Reputation
9,807
Next tag badge:
73/100 score
24/20 answers
Badges
1 7 32
Impact
~284k people reached

  • 0 posts edited
  • 21 helpful flags
  • 245 votes cast
11h
comment Is R being replaced by Python at quant desks?
I am spending some time with Rcpp this weekend. Thanks for the pointer.
17h
comment Is R being replaced by Python at quant desks?
And hence we look for ways in either R or Python to migrate part of our design and pricing framework to. I tremendously benefitted from this discussion already and you make lots of very good and above all informed points. Thanks a lot for adding so much value.
17h
comment Is R being replaced by Python at quant desks?
Agree fully that each requires different approaches and poses different requirements in general. However in the end of the day I and my team still needs to get our work done in our framework of choice. For visualization, for example, we peruse a C# Frontend that we equipped with massive parallelization capabilities, customizability, and the ability to make use of hardware based technologies. For parallel and async computing we also interface with different technologies which is precisely why we look for a framework that boasts strong capabilities in interfacing with other components.
1d
comment Is R being replaced by Python at quant desks?
what I need to better understand is the computational efficiency of the actual statistical and numerical procedures. Your data tables can be as fast as they want but if the actual visualization of time series in R gets on its knees when you throw a million or so data points at it then you have your bottle neck right there. Same goes for MC pricing. Is that clearer?
1d
comment Is R being replaced by Python at quant desks?
as well as pricing derivatives via Monte Carlo, PCA, or more mathematically involved PDE solvers on the other end of the spectrum. I get the point that indexed data tables allow for fast access to chunks of data but this only serves the starting point of any analytical or numerical exercise...
1d
comment Is R being replaced by Python at quant desks?
I thought I was very specific about my requirements in the question I originally asked. Just because some of the requirements require large data quantities and others do not should not be confused with me not knowing what I want. I look for a development and analytical testing architectural change that needs to cover both, the analysis and visualization of vast amounts of time series based data and options order book data on any end of the spectrum
1d
revised Is R being replaced by Python at quant desks?
added 376 characters in body
1d
comment Is R being replaced by Python at quant desks?
@statquant, I played a bit with the data.table and while it seems indeed significantly improve data table grouping and table transformations my original concern is not addressed. For computational efficiency the organization of input data is only one part of the equation. The main resource consumption will be taken up by the respective statistical and mathematical computations and that is where I am not (yet) sold that R comes close to Python's stats and math libraries in terms of computational efficiency.
1d
revised Is R being replaced by Python at quant desks?
added 212 characters in body
2d
comment Is R being replaced by Python at quant desks?
Statquant, I appreciate your offer and will contact you. I only mentioned kdb as you brought it up. I am definitely interested in gaining more insight into the data.table package you mentioned because performance in R was a deal breaker for me so far.
2d
comment Is R being replaced by Python at quant desks?
That grouping and transforming on 20 million rows takes less than 1 second as well as you stating that the speed reaches kdb performance benchmarks...
2d
comment Is R being replaced by Python at quant desks?
@statquant, that is a pretty bold claim you make. I am happy to whip up a few test batteries when I find time in the next couple days but being a kdb user myself I find that pretty hard to believe. I will report back with some numbers. Thanks for your answer and for sharing your insight. My intent to move away from kdb by the way is the precise reason that caused my asking this question.
2d
comment Is R being replaced by Python at quant desks?
Thanks for following up on this.
2d
comment Is R being replaced by Python at quant desks?
And when you talk about something crashing then the problem lies with improperly providing the required input format. The same can happen in OOP languages, Python and R.
2d
comment Is R being replaced by Python at quant desks?
Secondly datatables in R are very very slow. Throw a few million time series data points at it and data frames go to their knees. The only thing I have seen that was fast was an implementation that perused memory mapping. But one could argue this is just an interface R peruse ...as soon as you actually grab the data and run R functions over it becomes very slow. Caveat here: I have not looked at any new developments over the past 8 months in R space. If there is anything new I would be happy to be pointed to it.
2d
comment Is R being replaced by Python at quant desks?
Hmm I guess I need to disagree with you here regarding visualizations. R packages are still lightyears behind efficient and especially dynamic visualization. Every first year IT student can chart a time series from scratch. What people want and need is visualization of millions of data points that a charting app can down sample. Fast zooming and panning and handling of annotations. I have not seen anything in R that comes even remotely close.
2d
comment Is R being replaced by Python at quant desks?
what I meant was backward compatibility within the Python stack. Is it true that using Python 3.x unables me to use packages that target 2.x?
2d
comment Is R being replaced by Python at quant desks?
I did not notice a fury nor downvotes. And I fully agree with your suggestion. What really currently discourages me to again more actively participate on this site is pressure to conform to strict "rules" and guidelines. Humans are not bits and bytes nor does efficient and intelligent learning involve black and white Q&A formats. As this question demonstrates the format itself is already questioned because some seem to feel incredibly uncomfortable to go out of their "rules-based" comfort zone. I also like to see more healthy debate and sharing...
May
24
comment Is R being replaced by Python at quant desks?
Thank you for sharing your experience and providing pros and cons, I appreciate the balanced thought sharing. Though regarding backward compatibility, does Python 3.x not break backward compatibility? And in terms of mathematical and statistical features of packages, R clearly still has the lead here, imho. At the same time, however, I do not see much value in 90%+ R packages because they target a very specific statistical approach and the implementation is not modularized and not extendible, so that functionality remains very limited, almost to the degree of single time usage.
May
24
comment Is R being replaced by Python at quant desks?
@rhaskett, regarding inter-connectivity, I think it is extremely important to be able to efficiently interface with other modules, hardware, software applications. I believe it to be a myth that most who seriously perform data analyses do not need inter-connectivity. In that I find Python a lot more capable and it provides more efficient means to, for example, fan out computations to other hardware instances.