Friday, 11 January 2013

FPGA Smart NIC cards

If you haven't read the Lockwood etal 2012 [1] paper on High Frequency Trading (HFT) and Smart NICs you really should as it is one of the better public domain overviews.  Old hat for HFT types but a decent, if a little dated, overview of what was state of the art.

In Lockwood they give examples of "Smart NICs for Low-Latency Finance" in table 1 from three vendors: Dini Group, AdvancedIO and Accelize.

All three vendors offer good looking cards for PCI Express, SFP+s, and have PPS or GPS timing ports or options for such.  Not all the cards in the table are still available as the vendors have moved on and whilst some are still around they also have better and sometimes cheaper ones.  Mike Dini's group have introduced a nice Kintex one with PPS timing support for example.

There are plenty of other good vendors out there and here are a few that I'm aware of but I don't necessarily know their product:


Good dedicated packet capture tech from some of the vendors above (I use Endace and custom in-house cards) and also these guys and gals who aren't really in the transmit space AFAICT:


My company makes stuff like this too with a twist and slice of lemon, but there are many more vendors out there.  I'm interested in compiling a bigger list, so please send me firms that you may know of that make PCIe FPGA NIC boards so I may try to learn more about what products are out there.

FPGA's are not always the easiest to use and often you're better off using a fast network card without the FPGA hassle such as those low latency Ethernet NICs available from Mellanox, Solarflare or Myricom.  At least kernel bypass has been around for over twenty years now and you should be using it. The real thought is that when you can do 1.3 ~ 2.5 microseconds wire-to-wire with such COTS NIC cards you have to think long and hard about the hassle of an FPGA...

____________

[1] J. W. Lockwood, A. Gupte, N. Mehta, M. Blott, T. English, and K. Vissers, “A Low-Latency Library in FPGA Hardware for High-Frequency Trading (HFT),” 2012 IEEE 20th Annual Symposium on High-Performance Interconnects, pp. 9–16, Aug. 2012.

7 comments:

  1. http://info.iet.unipi.it/~luigi/netmap/ -- not exactly in the same league but even more simplified (in that it uses select-able file descriptior and optionally ioctl), claims to easily saturate 10G link (in terms of bandwidth I guess)... no figures on the latency though (I must admit I did not look very closely though)... would be interesting to experiment-with...

    ReplyDelete
  2. Thanks for that. Yes NetMap shipped in FreeBSD 10.0 in Jan 2014 FWIW. That kernel bypass movement started in the early nineties, perhaps earlier but I have no papers earlier than about '92. Myrinet / Myricom was one of the earliest practical proponents. Usings these kinds of efficiencies tools like zmap are improving as a nice by product. Zmap can now do a full scan of the ip4 addresses in under an hour showing that some of the apps are catching up in their thinking too.

    I think lower latency non-PCIe approaches for connecting to the CPU offer the best promise for network cards competing against dedicated hardware for all but the most demanding applications.

    ReplyDelete
  3. There have been many different terms and concepts associated with TCP offload and one of them is the 10G Bit TCP offload which has a significant role played in the entire process. It comes with unique and amazing features and benefits making it an exceptional offload engine. There are actually several interesting points to know about
    10G TCP offload.

    ReplyDelete
    Replies
    1. There are a few TCP FPGA engines around now. Dini group is marketing one too. Maybe we should prepare a list of them?

      Delete
  4. NIC with Full TCP Offload technology mainly used with high-speed network interfaces like 10G Ethernet for offloading the entire TCP/IP stack to the network controller |NIC with full TCP/UDP Offload|
    TCP Offload, Full Kernel Bypass

    ReplyDelete
    Replies
    1. Thanks for the second reference to Intilop. Do you still have the pricing model around tcp connections? I know a few vendor used to package Intilop or recommend it. Always a bit expensive for me so I don't know too much about Intilop.

      TCP in hardware is getting to be quite a busy space now. How does Intilop compare to others, such as Mike Dini's? ComBlock is offering TCP client & server IP for under $1,000, any thoughts on that one? PLDA are also offering Xilinx and Altera TCP. Also Fraunhofer has one of the original TCP hardware IP stacks FWIW but it was also expensive some years ago. The OpenCores work doesn't seem up to scratch in the TCP department (yet), but hopefully it is just a matter of time...

      Delete
  5. Its a great post with perfect information aboutSmart NICs. Thank you for providing this Article.

    ReplyDelete