What are the current computational (non-network) bottlenecks now in a quant's workflow? What computational tasks would be revolutionary with a 10-100x improvement in performance using general purpose GPUs?

  • 2
    $\begingroup$ This question is a bit broad. We're trying to avoid survey questions during the beta. Can you rephrase it to a specific issue that you face? $\endgroup$ – chrisaycock Feb 14 '11 at 21:26
  • $\begingroup$ I agree. Could be an interesting question if it was a little more focused. You tagged it gpu...are you asking what kind of quant problems can be improved by using a gpu? $\endgroup$ – Shane Feb 14 '11 at 21:32
  • $\begingroup$ I develop GPU codes, but I wanted it to be of general interest. We computational scientists need problems to work on :) $\endgroup$ – Chad Brewbaker Feb 14 '11 at 21:37
  • 2
    $\begingroup$ A hammer looking for a nail, huh? You know how many options pricers I've seen in GPU and FPGA variants because of that? And I don't even trade options! If you're really that curious, the biggest non-network bottlenecks are memory-bound operations, like OLAP. The CPU is rarely the bottleneck. $\endgroup$ – chrisaycock Feb 14 '11 at 22:33
  • $\begingroup$ en.wikipedia.org/wiki/Online_analytical_processing Interesting... actually I am working on some multidimensional spacial query algorithms... thanks! $\endgroup$ – Chad Brewbaker Feb 17 '11 at 6:10

Coming from an HPC background myself, I know too well the feeling of owning a hammer and yet having no nail. Your question is about computational bottlenecks that can be relieved with GPGPU, though I'm afraid to admit that there aren't many in finance. For realtime applications, the network is the bottleneck; for historical applications, the memory is the bottleneck. The CPU is rarely saturated in my line of work.

However, there is one particular area that does appear to be CPU bound: interpretation. Namely, the feed handler and the FIX parser both require many small amounts of data to be transformed from one representation to another. FPGA-based feed handlers are starting to become more popular; I haven't seen anything similar for FIX parsers though.

If you could show how to parse a FIX message with a GPU off the wire, then that might be interesting. FIXT 1.1 can support InfiniBand, so NVIDIA / Mellanox's GPU Direct set-up would be especially noteworthy, though not required. (There aren't many trading venues supporting FIXT right now anyway, so there's no rush there.)

If you wanted to generalize your work for all key-value pairs communicated over a network, you might be able to apply some of your findings to parsing HTTP headers in realtime. No doubt many cloud vendors would be pleased to see that.

By the way, the reason I advocated FIX parsing instead of feed handling is that most data vendors ship their own proprietary API. Good luck getting Wombat to cooperate with you until you have some results of your own to show.

  • $\begingroup$ Similar problem in computational biology with parsing FASTA files. Is quickfix a decent code base to start from? quickfixengine.org/index.html $\endgroup$ – Chad Brewbaker Feb 17 '11 at 6:27
  • $\begingroup$ @Chad QuickFix is a decent reference implementation, though there are much faster systems for specific needs. The fastest shops out there build their own custom library in-house. $\endgroup$ – chrisaycock Feb 17 '11 at 6:36

In exotics options pricing, there are lots of CPU bottlenecks -- for example the calculation of Fast Fourier Transform or Monte Carlo simulation. When I price a range accrual in Libor Market Model, I don't use a lot of data (carefully optimized, everything should fit in a few MB of L2 cache), but I do a lot of calculations. This is where, I think, a GPU may be useful.

  • $\begingroup$ I figure exotics would do more calculations than my domain (cash equities). What's your CPU utilization at normally? $\endgroup$ – chrisaycock Feb 15 '11 at 14:09
  • $\begingroup$ When I'm pricing? Hovering around 100%. $\endgroup$ – quant_dev Feb 15 '11 at 15:22
  • 1
    $\begingroup$ Get this man a GPU, stat! $\endgroup$ – chrisaycock Feb 15 '11 at 17:20

Not the answer you're looking for? Browse other questions tagged or ask your own question.