13
$\begingroup$

I have read about different implementations of HFT systems on FPGAs.

Vendors like Cisco claim they have achieved the same results with high performance NIC's (http://www.cisco.com/c/dam/en/us/products/collateral/switches/nexus-3000-series-switches/white_paper_c11-716030.pdf).

My question is, what part of HFT systems are mostly implemented on FPGAs nowadays? Are FPGAs still very popular? Is only the feed handler implemented on the FPGAs? Because some of these systems described above only have a feed handler implemented on the FPGA, because the strategy changes too much, or is too hard to implement on FPGAs. Others claim that they have also implemented trading strategies on FPGAs or using high performance NICs instead of FPGAs to build HFT systems. I've read about different approaches but I find it hard to compare as most of the results are tested on different input sets.

$\endgroup$
1
  • $\begingroup$ There are more uses of FPGA's than the ones you mention. An FPGA board often sits where the NIC does in a CPU-only system. In some cases the FPGA is nearly stand-alone, receiving market data, calculating a strategy's theos, firing orders and hedging fills. At the other end of the scale they work as a smart NIC, pulling in the raw market data and order traffic and feeding an application running on the server with only the data it needs. An FPGA or NIC has similar transceiver latency to the network as it does over the PCIE bus, but 2x as more for everything that goes to the app. $\endgroup$ Commented Feb 26, 2018 at 19:48

7 Answers 7

15
$\begingroup$

Here's a way to think about it: imagine you can do something in an ASIC (i.e. directly in hardware). However, the process of fabrication is in itself expensive, and you get a design that you cannot change afterwards. ASICs make sense for predefined tasks such as Bitcoin mining, well-known data processing algorithms, etc.

On the other hand we have ordinary CPUs (as well as coprocessor CPUs and GPUs) which are general-purpose, but process a small (in terms of concurrent instructions) set of instructions at a very high speed.

FPGAs are the middle ground. They are 'hardware emulators' and as such can be considered to be 10x slower than actual hardware, but still way more performant for concurrent operations than CPUs provided you are able to utilize the die to spread your logic accordingly.

Some uses of FPGAs are:

  • Video transcoding (e.g. HD video decoding in TVs) as well as various data acquisition boards
  • Fixed data structure parsing (Regex parsing)
  • Discrete system simulation (for example, simulating the outcome of a card game)
  • Lots of 'properly embedded' applications such as e.g. in aerospace or scientific research

The problem with FPGAs for quant uses is that it's not so good for floating-point calculations, particularly since ordinary CPUs are already optimized for that with things like SIMD. However, for anything fixed-point or fixed-size data structures, FPGA design allows you to configure the device to do a lot of processing at the same time.

Some things done in trading are using FPGA for feed handlers (parsing directly from the network stream) as well as building certain parts of the trading structure (e.g. order books) in hardware in order to be able to deal with the rapidly changing data structure without loading the CPU.

FPGAs mainly aim to address the concern of quickly processing data without paying the propagation costs. This is particularly in contrast with devices such as GPGPU (or any PCI-dwelling card, such as Xeon Phi) which pay performance penalties for getting data to/from the device. That said, DMA options are improving in this regard, too.

$\endgroup$
5
$\begingroup$

FPGA's are really nothing more than the same logic blocks repeated again and again throughout the silicon, with configurable switches to connect the logic blocks together. This makes FPGA's very good--and fast--at dealing with repetitive problems that can be described in a hardware circuit that does not change during operation. And you can have literally thousands or tens of thousands of these circuits, all working in parallel at the same time, in just one FPGA.

CPU's on the other hand are based around the ALU, which loads instructions, loads data, operates on the data, maybe stores the results, and then does it all over again. CPU's then are very good--and fast--at dealing with problems that are continually changing--both in size and in scope and at switching between different tasks. Today's CPU or core will have tens to hundreds of ALU's with parallel pipelines for data and instructions, which makes them very fast at complex problems that can be worked on in parallel.

These designs make FPGA's faster at simpler problems that can be attacked with a vast parallel architecture--such as condensing down multiple data feeds in less than micro-second, wire-to-wire, or triggering a pre-calculated buy, sell or cancel on a price that matches a particular pattern. CPU's are faster at more complex problems that require less parallelism, such as calculating the basket of buys, sells and cancels needed to keep a portfolio risk-adjusted or integrating a number of price and news sources of varying age and quality into trading indicators used by traders and management to decide what adjustments they will make to the trading system.

Where FPGA's are used in HFT depends a lot on the architecture of particular shop. They are best used performing simple, repetitive, wide tasks and performing them quickly. CPU's are a Swiss knife that can do most anything, especially where the requirements are changing and the dimensions of the problem are not fully understood at the outset.

$\endgroup$
2
  • $\begingroup$ Whats the advantages of FPGAs over GPUs then? $\endgroup$
    – Trajan
    Commented Feb 25, 2018 at 18:52
  • $\begingroup$ @Permian a GPU is very similar to a CPU, but it is better than a CPU at certain types of problems especially those that are compute intensive or involve math. While a GPU can have thousands of cores, an FPGA can have tens of thousands or hundreds of thousands of independent threads executing simultaneously, though each FPGA thread is executing much simpler code than a CPU or GPU. $\endgroup$ Commented Mar 11, 2018 at 22:30
2
$\begingroup$

Your question really makes not much sense. It's like asking how much of the wiring in trading infrastructure uses optic fiber and how much of it uses copper. The best answer that we can give to you is that an FPGA is not a magic bullet.

Vendors like Cisco claim they have achieved the same results with high performance NIC's (http://www.cisco.com/c/dam/en/us/products/collateral/switches/nexus-3000-series-switches/white_paper_c11-716030.pdf).

This is an incorrect interpretation of Cisco's white paper. There's very little overlap between the use cases of switching fabric and those of a FPGA.

what part of HFT systems are mostly implemented on FPGAs nowadays?

Currently, FPGAs are often used in our printers and TV set-top boxes.

$\endgroup$
1
$\begingroup$

I want to highlight the “digital signal processing” (DSP) block with ALUs. Today’s FPGAs have hundreds of programmable DSP blocks – the very largest having thousands.

Now, suddenly, you have thousands of small processors at your disposal, all able to perform calculations in parallel. This is vastly in excess of parallelism provided by the Xeon Phi or GPUs. In fact, if you’re doing options price modelling or stochastic risk modelling on FPGA, you can get more than 100-fold increase in performance compared to the latest GPUs and even more compared to the latest CPUs.

Along with the DSP blocks, the other major factor in this performance gain is the memory cache. FPGA has built-in distributed RAM that is extremely fast, allowing bandwidth of 100TB/s to be achieved at datapath level.

Using today’s FPGAs for algo strategies gives large and massively concurrent compute resource that is able to give 100 to 1000 fold increase in performance compared to GPUs or CPUs. The main caveat is that you would have to become proficient in writing in Verilog or VHDL :)

Sanjay Shah CTO Nanospeed

$\endgroup$
1
$\begingroup$

A variety of powerful, many-core processors are beginning to make their way into the the hardware acceleration space that was previously completely "owned" by FPGAs. Companies like Tilera, Adapteva, and Coherent Logix all provide these processors here in the US, with Enyx from France also making inroads.

The true measure of effectiveness of these massively parallel processors lies in the maturity of their software tools. That's where the prospective user should focus their attention. Nobody wants to program or debug tens or hundreds of cores using manual techniques. Of course, it goes without saying that I/O bandwidth is as important.

In my personal experience in this space I'm seeing customer adoption of Coherent Logix processors as co-processors or hardware accelerators for C-language algo acceleration. By enjoying the rapid design cycle of a C based environment, algo programmers can tweak code to their hearts content and not worry about costly and time intensive HDL coding for FPGAs.

The optimal partitioning is to have FPGAs do what they do best - fixed repetitive operations - and have many-core processors do what they do best: accelerate algo developers productivity and execution speed.

John Irza, Business Development Manager, Coherent Logix, Inc.

$\endgroup$
0
$\begingroup$

Nearly all HFT shops use FPGA architecture. These devices need to be replaced frequently as the quickly are outpaced by the latest improvements in speed, pipelines, parallelism, etc. Unless you are ready to invest $2M a year, figure out another strategy. Lot of guys doing daily price moves with pen and paper are making billions in Omaha, NB.

$\endgroup$
0
$\begingroup$

They are implemented based on:

  • high-level synthesis, or behavioral synthesis
    • behavioral/high-level models based on SystemC (a modern HDL based on C++), C++, or C, or even MATLAB are synthesized into register-transfer level (RTL) models in hardware description languages (HDLs) or digital circuits (structural models in HDLs, or logic-level representations).
    • This is the fastest way for financial engineers, especially those with skills in computational finance but lack skills in digital integrated circuit (IC) or VLSI system design, to implement algorithms for computational finance on FPGA platforms to exploit hardware acceleration. If they can program in C, C++, or MATLAB, they can use this approach.
    • National Instruments has a high-level synthesis tool for FPGA platforms, using LabVIEW. However, this is a very costly option. Student licenses for LabVIEW can cost US\$1,000.
  • other high-level models based on hardware construction languages (HCLs), such as the following HCLs that are categorized into the programming languages that they are based on:
    • Scala: Chisel HDL, from UC Berkeley
    • Python: PyMTL from Cornell, PyLog from UIUC, PyCDE from CIRCT + MLIR/LLVM, PyRTL from UC Santa Barbara, MyHDL from Europe, and PyGears from Serbia
    • Haskell: Clash, from the University of Twente (the "l" is typeset as $\lambda$ in LaTeX)
    • OCaml: Hardcaml, from Jane Street Capital
    • The caveat is to know the programming language that the HCLs are based on, so that you can exploit them in the HCL models to create succinct models rather than the more verbose models in Verilog/SystemVerilog or VHDL.
      • E.g., the founding developers of the RISC-V instruction set architecture (ISA) used an agile hardware approach, which is analogous to agile methods in software development like Scrum and extreme programming (XP), for VLSI implementations of their RISC-V processor architecture designs. On average, they used about 100 lines of Chisel HDL code to represent 1000 lines of Verilog code. However, they exploited functional programming and object-functional programming to achieve such productivity gains. Hence, I mentioned Clash and Hardcaml at the bottom, because who really wants to learn Haskell and OCaml, just to design digital ICs, or implement algorithms on FPGA platforms. If you work for Jane Street Capital, and have implementations on OCaml, it is easier to map the OCaml source code to Hardcaml models. Else, good luck!!!

Notes: Coarse-grain reconfigurable architectures (CGRA) provide an alternative to fixed digital IC designs based on custom digital IC designs (i.e., manual IC layout designs), standard cell -based digital IC designs, and FPGA platforms.

The trend of hardware/software co-design for domain-specific computing, which optimizes domain-specific languages (DSLs) for domain-specific architectures/accelerators (DSAs), is gaining traction in the research and product development communities.

Hence, not all trading systems would be leaning on FPGA platforms. GPU platforms and hardware accelerators for machine learning are far more popular.

$\endgroup$
2
  • $\begingroup$ Is this answer AI-generated? $\endgroup$ Commented Oct 22 at 13:21
  • $\begingroup$ No. I do research on system-technology co-optimization, and develop electronic design automation software to automate the design of electronic and computer systems. $\endgroup$
    – Giovanni
    Commented Oct 23 at 2:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.