▲ 1 points

Ask YC: How to Build a High Performance Net Traffic Recorder With 0 Bottlenecks

by tzury·16y ago·3 comments·view on hn ↗

I need to build a server which is capable of capturing 20 Gigabit/second network traffic (2 x 10g network adapters - 99% utilization) and store them on a disk with zero packet-lose. The requirements is supply the ability to record 20-30 minutes of continues traffic.

Not being an “Hardware Expert”, I have search the net for today’s fastest hard drives and came up with the following option: Setting Raid-0 (stripe) over 6 x 600 GB WD’s VelociRaptor SATA drives, which claim to be the fastest (sata) drive on earth (6 Gb/s interface) - that would provide 36Gbps writing speed.

My questions are basically, having the hard drive spot sorted out, what about the other parts of the machine, what kind of a motherboard would I need? What kind of a CPU? How many CPUs? How much RAM is needed given the fact data is not manipulated but written into the hard drives.

1) WD Drive details available at http://wdc.com/en/products/Products.asp?DriveID=821

So, you basically want to capture 9TB of network traffic (2x10GBx30Minutes).

Does it have to be on a single machine? You may have better luck splitting the task across 2 machines.

What NIC are you using?

I would think I'd start with a test run of a 64bit linux build on a quad-core box with just "tcpdump -i en0 -s 2000 -nn -w text.cap" and see what you get.

Either Intel's Gen2 adapters or others, not sure yet.

WD Drive details available at http://wdc.com/en/products/Products.asp?DriveID=821