Most quantitate libraries use float64 precision for monte-carlo or other method. Some academic papers do experiments on float16 and find it has some restrictions on float16.
I just wondering if float32 precision is enough in industry?
$\begingroup$
$\endgroup$
5
-
$\begingroup$ What do you expect to gain from using float32? $\endgroup$– AKdemyApr 6 at 22:04
-
$\begingroup$ @AKdemy on some accelerators, for example TPUs and GPUs, float32 is quite common than float64. If float32 is enough, then I could run on these accelerators. $\endgroup$– luweizheng36Apr 7 at 5:47
-
1$\begingroup$ Why don't you just compare the loss of precision due to using 32bit with the gain in precision due to larger samples? $\endgroup$– g gApr 7 at 7:41
-
$\begingroup$ If float32 were not enough then -according to this- a lot of option prices pre 2003 (the year when 64 bit CPUs were introduced to the mainstream personal computer market) must have been wrong . $\endgroup$– Kurt G.Apr 7 at 19:04
-
$\begingroup$ @KurtG. This is not the case, the float64 aka the double was already in widespread use before the switch to 64bit. The change is mostly about the size of pointers which allow addressing 2^64 bits of memory instead of 2^32 but there is more as described on Wikipedia. $\endgroup$– Bob Jansen ♦May 13 at 18:39
Add a comment
|
1 Answer
$\begingroup$
Usually
If you want the accuracy of
$\endgroup$
Usually float32 is enough
For most applications in finance, float32 is plenty of precision. This is generally suitable for naively achieving about 5-6 significant figures of accuracy. There are so many sources of error, that to trust results beyond this are moot. Consider for example:
- The model error (the Black-Scholes model is clearly wrong).
- The data has noise (market data is both noisy and usually only quoted to a few significant figures).
When you really want accuracy
If you want really high precision, either for a more theoretical (and less data driven) options pricing question, then it may be useful to switch to high precision. However, in many cases the average accuracy of single precision simulations gives (after averaging) a much greater accuracy than the constituent parts.
For many options pricing problems, note that a double precision accumulator is often required (e.g. to avoid numerical overflow).
If you want the accuracy of float64 but the speed of float16
For Monte Carlo based methods, it is possible (under certain conditions) to utilise low precision formats such as float32 or float16 or even Bfloat16, but still retain the accuracy of a much higher precision. How? Formulate the problem as a nested multilevel Monte Carlo. The simple explanation of this can be phrased as "Do lots of imprecise calculations very fast, and then do a handful of high accuracy corrections as negligible cost".
My PhD research was in exactly this topic, and I have written an article on interleaving numerical precisions in exactly this way to speed up Monte Carlo applications. See the following article:
Rounding error using low precision approximate random variables
As an example this article demonstrates how to achieve a 10-12 fold speed improvement without any loss in accuracy by mixing precisions in the calculations. It also takes into account how rounding error propagates in Monte Carlo simulations, and how to ensure this is correctly taken care of.
There are also a few companion articles about how to achieve further speed improvements if by similarly relaxing the precision of the random numbers in the right way and in the right places.
