Packet Capture at Scale: Planning Storage and Retention

Packet capture is not about collecting more data. It is about collecting the right data, for the right duration, with predictable performance and cost. Organizations that treat packet capture as an afterthought often discover the problem too late: storage fills up, storage bills mount, investigations lack sufficient historical context, or compliance questions cannot be answered with confidence.

This article explains how to plan packet capture infrastructure at scale, focusing on realistic volume estimation, ingestion and compute requirements, and retention strategies using the IOTA platform.

Start with a realistic traffic estimation

Storage planning starts with understanding traffic behavior, not link speed. A 10 Gbps or 100 Gbps link rarely runs at line rate continuously, but burst behavior matters as much as sustained averages.

Key questions to answer:

• What is the sustained average throughput per capture point?
• Size of peak burst rates?
• How many capture points run simultaneously?
• Is traffic symmetric or directional?
• How much of the traffic is actually relevant?

Packet capture volume is best expressed as data per day, not bits per second.

Example

If a link averages 2 Gbps sustained:

• 2 Gbps ≈ 250 MB per second
• ≈ 21.6 TB per day

At this scale, small planning errors quickly become expensive.

Ingestion matters more than analysis

At scale, capture systems fail more often due to ingestion bottlenecks than analysis limitations. Compute sizing must account for:

• Sustained packet ingestion at peak rates
• Indexing and metadata extraction
• Concurrent searches during active capture

The IOTA 100 CORE is designed to keep capture reliability independent of the analysis workload, which is a critical capability. Dropping packets renders any subsequent investigation useless. By integrating high-throughput ingestion with customizable capture policies, IOTA ensures storage and compute resources can be precisely matched to your unique requirements.

Retention is a policy decision, not a disk decision

Before choosing storage size, define:

• Why is it needed
• What data to keep
• Who can access it
• When the data is no longer needed

Capture options with Profitap IOTA

Using IOTA capture management, you can define whether to capture full packets, filtered packets, metadata only, or a combination.

You can apply filters to optimize the use of available storage, like:

• VLAN
• IP ranges
• Protocols
• Industrial traffic types
• Payload inclusion or exclusion

IOTA 100 CORE retention tables

At a sustained 100 Gbps, the IOTA 100 CORE produces roughly 1,080 TB of data per day (decimal units). That works out to about 1.33 minutes of full packet capture per terabyte of storage. In practice, full packet capture retention (hours) ≈ storage (TB) × 0.0222. Using the same proportional assumptions as the original model, filtered packet capture retains ~20× longer than full packet capture, and metadata-only retains ~100× longer.

Typical capture strategy

Most large deployments combine all three modes:

Retention governance and audit readiness

A defensible retention policy is crucial to mitigate risk and requires robust governance. Key components include:

• Establishing specific retention periods for each packet capture type.
• Defining clear ownership and implementing necessary approval workflows.
• Ensuring controlled access mechanisms for all raw packet data.
• Implementing verifiable processes for data purge operations.

Putting it together with Profitap IOTA

Profitap IOTA platforms are engineered to provide predictable, large-scale packet capture. By integrating IOTA with passive fiber TAPs and flexible capture management, users can achieve:

• Lossless, high-throughput traffic capture
• Storage capacity that scales with organizational needs
• Support for various retention policies

Planning assumptions used

These are conservative, standard planning values:

• 1 Gbps sustained capture ≈ 10.8 TB per day
• RAID 5 usable capacity assumed
• 50/50 average capture/analysis balance
• No compression

Packet capture at scale is not about storing everything forever. It is about building visibility that remains useful, affordable, and trustworthy over time.