# Building custom indices; getting data from web; stats analysis; Python or R?

I would like to build a couple of custom indices. I would like to be able to enter ticker(s) into an input and have ohlc, volume, qualitative ...data downloaded from yahoofinance, google finance, finviz etc over x period. From this I would like to build a geometric average indices for high momentum stocks and value stocks. I would then like to conduct analysis of these indices as they relate to each other. What stocks have highest/lowest correlation over x period, volume/range analysis, momentum over x period, sma for pairs trade....Is this a job for python or R? Do you have any suggestions on what packages/resources I would need to do this kind of analysis? I appreciate your help.

In addition to the above answers - You should be very careful that you do not introduce survivorship bias in your creation of indices and choose your data source carefully to remove such bias. For example, Yahoo Finance only contains currently-listed securities.

• Great point. I'm using IBD's top 200 and bottom 200 to form the indices. I was planning on choosing an arbitrary number 1000 for a starting point. By their very nature they rotate names in out of the top/bottom 200. There isn't that much turnover but was wondering if that ruins my whole idea? – atrain Mar 31 '15 at 22:48
• Even though you think there might not be much movement... there is. For example, in the Russell 3000 there have been 92 removals (delistings) from the Russell 3000 since reconstitution last June 2014 or around 3% of the constituents. Also, there have been 89 IPO additions since the June 2014 reconstitution (as at mid Mar 2015) - these IPOs are added to replace those stocks delisted. – Norgate Data Apr 1 '15 at 1:03
• Well there is certainly going to be more rotation in the IBD selections than the Russ. Is there going to be a huge problem with the rotations in and out of the index? There isn't going to be any market cap weighting to the the indices. – atrain Apr 1 '15 at 17:52

Both R and Python can do this very nicely.

For Python you would need the pandas package and its dependencies. pandas has a lot of basic statistics, but for more advanced statistics like it looks like you want to do, you can use the statsmodels package, which can work directly with pandas data types. It can also download the csv files directly off the website if given a url, even from https sites. Further, it can download the sort of stock data you want for you, just by giving it a stock ticker and a date range. You can download a python distribution like anaconda or python(X,Y) which will have pandas and statsmodels built-in, so no additional installation is necessary.

R doesn't need any additional packages. It can do roughly the same things as pandas and statsmodels for your purposes. It can also download csv files off the web if given a url, but apparently chokes on https files (which pandas doesn't), although you may not even be downloading any files through these programs. You can use another tool to do this in R, though, and it will likely only add an additional line or two of code. With additional packages, such as quantmod or Quandl, it can also download stock data using a ticker and date range.

• It sounds like they are both up to the task. I plan on incorporating the analysis into a website. Does this make python the clear winner. The only reason I ask is that there seems to be significantly more resources on R as it relates to Finance than Python. – atrain Apr 1 '15 at 17:55
• You are probably better off with Python and pandas, then. You can get another package that lets you output R in html, but this is another thing that pandas has built-in, using the very high-quality HTML tools available for Python. Python on its own does not have the statistics tools that R has, but to do what you want to do it requires adding several new packages to R, so it would be easier to just download a Python distro that has this all built-in. Further, Python is easier to integrate with existing web technologies. – TheBlackCat Apr 2 '15 at 9:34

It is essentially a statistical exercise, so I would choose R.