0
$\begingroup$

I want to download historical data for different indices. I am using Python for this. I used the following code.

import pandas_datareader.data as web

start = datetime.datetime(1960, 1, 1)
end = datetime.datetime(2020, 4, 30)

SP500 = web.DataReader(['sp500'], 'fred', start, end)

It gave me data from 2010 onwards. 

SP500.head()

              sp500
DATE               
2010-05-03  1202.26
2010-05-04  1173.60
2010-05-05  1165.90
2010-05-06  1128.15
2010-05-07  1110.88

Is there a way to pull the complete history? Also is it possible to pull data at daily/weekly/monthly/quarterly frequency. It does not have to be datareader, it can be any other function that can accomplish this.

Improve this question
$\endgroup$
1
  • $\begingroup$ If this is a repeat question, please point me towards any previous answer. $\endgroup$
    – deb
    May 2, 2020 at 15:15

1 Answer 1

Reset to default
1
$\begingroup$

The solution here is very simple. At any point in time, you will be able to retrieve from the internet stock market data at a daily frequency for free, as long as (1) you do not mind a possible delay of a few hours and (2) having only the past few years of data.

So, all you need to do is to find a dataset today that spans the missing time-frame. Then, you save it somewhere from which you can always retrieve it. Then, you can periodically update this old dataset so that, in practice, you only always need to download the last few days from the internet.

EDIT (following suggestion by noob)

I checked and you can go back to 1927 on Yahoo!Finance: https://finance.yahoo.com/quote/%5EGSPC/history?period1=-1325635200&period2=1588377600&interval=1d&filter=history&frequency=1d

OLD PROPOSAL

I don't know how far back you can get, but I know that Christian Dorion offers his code and his data for an option pricing paper he just published on his website. Buried in his data files are .mat (MATLAB data files) files containing among other things the S&P500 going back at least to 1990. It's not going back to 1960, but it's a good start. Combining this with FRED, averaging 252 trading days per year, that would make up around 7560 data points. Here.

Improve this answer
$\endgroup$
5
  • $\begingroup$ You can easily find online the S&P going back to 1950 or 1960, just Google. $\endgroup$
    – nbbo2
    May 2, 2020 at 19:14
  • $\begingroup$ I don't know why, I just assumed you couldn't get that data. You can go all the way back to 1927 on Yahoo!. $\endgroup$
    – Stéphane
    May 2, 2020 at 19:23
  • $\begingroup$ I was looking for a way to automate that process in python. I agree that I can always save csv file and then load it. I am looking for a way to automate that. And this is not specific to SP500. I want to be able to do it for all possible indices, where if I give the appropriate ticker, startdate, frequency, I get the data back. $\endgroup$
    – deb
    May 3, 2020 at 3:00
  • $\begingroup$ My point, @deb, is that you need only to automate the most recent updates. Some of the data you might want could only be available for just a few years for free. It is fairly possible that you might have to combine many data sources to go back far in time. But it's a none problem because you only need to get the distant past once -- it's only the newest data that you need to append. Get it? $\endgroup$
    – Stéphane
    May 4, 2020 at 2:17
  • $\begingroup$ Thanks. Got it. $\endgroup$
    – deb
    May 4, 2020 at 4:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.