8

Systematically finding most liquid futures instruments Can we put together a better list than the academic articles? Yes! The lists in existing publications [1, 2] are great, but fall slightly short of your goal: I'm asking for a systematic, repeatable procedure for determining a list of what I expect to be around 100-300 markets instruments. [3] What'...


7

This issue is incredibly important and I agree there is little practical information about it. To me, the key idea is to find the right matrix completion algorithm that best suits your needs. I work mostly with equity time series and there are substantial missing values issues due to, e.g., as you cite, IPOs with limited history. Recently I have had good ...


4

Welcome. You're looking at 1995 data. Back then, Edgar was just coming online. They did not have documents in electronic form. If you want data that old, you may have to pay a vendor, such as S&P. If you look at the same Макдак for 2019 https://www.sec.gov/cgi-bin/browse-edgar?action=getcompany&CIK=0000063908 , everything can be downloaded.


4

The actual term for this is downsampling. If you are trying to take 1-minute bar data and create 5-min bar data there are a few rules you should follow. The bar name is where it starts, not where it ends. The bar in your example should be called the 15:00 bar, not the 15:05 bar. It should not include the 15:05 1-minute bar. The bar should start at 15:00 ...


3

I am not sure I perfectly understand your question, the concept of "time series with varying density over time" is not very clear. One thing is for sure, the optimal way to "feed" a neural network is a function of the type of NNet itself and of the learning method you have chosen. For time series either you believe your data are iid vectors, and you can ...


2

I will prefix my answer with the following: I've not read the holy wars stuff in the other answers in detail. I think its missing the point. I've implemented exactly this project more than once in small buy side firms. Your main issue is getting the data in the first place. I have a recommendation - S&P Capital IQ. You can get a deployment where you ...


2

Both datagrapple and assetmacro are fake sites that collect your email address and password. Don't be fooled, CDS data isn't free.


2

For more granular (tick by tick trades, full order book L2 data) try https://tardis.dev - it's an API I've build out of need of such service for my own algo backtesting.


2

CRSP will give you returns with dividends and without dividends from SPX which allow you to compute those. If you do not have access you can use the data from this paper: "On the Importance of Measuring Payout Yield: Implications for Empirical Asset Pricing," Boudoukh, Michaely, Richardson, Journal of Finance, 2007. The data is freely available here: http://...


2

I think the issue can be addressed in two ways: statistical approach; economic approach. While I agree that ML/AI and other statistical tools can enhance missing data in time series, these lack economic meaning. One can implement these techniques and generate some numbers for the simulation. However, the derivation and the end result should also be ...


2

You could consider using the list of "liquid futures contracts" used in some previously published paper(s) on this subject, there are many. Alternatively, if you think previous studies missed some important contracts you could try to establish your own list independently. I thought for example of the using the following from a well know paper: Moskowitz, ...


2

As you'd alluded to, selling the data underlying their indexes is the primary way index providers make money aside from ETF licensing. About the best you'll be able to do for free is current holdings with weights--most providers offer a top N list as well as a download of current holdings for most of their indexes. This will likely come with some ...


2

If you have an Interactive Brokers account, you can get historical intraday index data, including SPX, through their API. Many developers find using the Interactive Brokers API to be a challenge, especially for collecting large amounts of data. If you want a more turnkey access, you can check out QuantRocket, which provides data collection tools on top of ...


1

You should forfeit developing any sort of trading strategy based on level 2 data if its purely based on the belief that the level 2 book contains additional information of value over level 1. First of all, your initial backtest design will, without a doubt, be highly susceptible to being flawed, making you think you accomplished something when in fact it'...


1

Bloomberg has float information, but it's imperfect. I imagine other vendors offer the same, but the reality is this sort of thing is typically done at the report level by hand (ie, analysts go through financial reports to determine % of shares held by 'insiders' (definition varies)). I'd be highly skeptical of any free source claiming to offer this ...


1

The solution here is very simple. At any point in time, you will be able to retrieve from the internet stock market data at a daily frequency for free, as long as (1) you do not mind a possible delay of a few hours and (2) having only the past few years of data. So, all you need to do is to find a dataset today that spans the missing time-frame. Then, you ...


1

If you are looking for a quick and easy solution, I have found a combination of Google Sheets + Yahoo Finance URLs to be relatively easy to implement. Here's an example you could try yourself. Let's say you have a stock ticker, "AAPL" in cell A1. =INDEX(IMPORTHTML( CONCATENATE("https://finance.yahoo.com/quote/", A2,"?p=",A2,"&.tsrc=fin-srch") ,"table"...


1

A widely accepted method to estimate Beta is the Vasicek (1973) method, which computes a preliminary estimate of Beta by linear regression and then "shrinks it" (adjusts it) towards 1 to compensate for the fact that the OLS Betas tend to be too extreme (too far from 1) in the cross section. I consider it the standard. Recently Ivo Welch has published a new ...


1

Take a look also at HistData.Com, they have both 1 minute data (which I use) as well as Tick Data. It is free or very inexpensive depending on the method of downloading you choose.


1

I've built an Excel Add-in (https://www.excelpricefeed.com/) which enables retrieval of data from Yahoo Finance (as well as other data providers). The Add-in currently provides live and historic prices to Excel cells via simple formulas such as: =EPF.Yahoo.Price("AAPL") =EPF.Yahoo.Historic.Close("TSLA", "1 May 2019") It also has a UI for bulk downloading ...


1

If you are just looking for basic intraday data (open, high, low, close, and volume data), you can check out Alpha Vantage. File can either be in json or csv format. They provide 500 API requests per day. If you require a higher API volume limit and technical support, you need to sign up for their premium membership. Another API-based data vendor is IEX ...


1

There are some decent historical intraday datasets at FirstRate Data - 15-20 years of 1-minute intraday data as well as tick data.


1

You can fetch Indian stocks data from NSEpy.The data available on NESpy are: 1. Daily stock data 2. Stock futures data 3. Stock options data 4. Index futures data 5. Index options data Example to get daily stock data from NSEpy in Python: from nsepy import get_history from datetime import datetime start = datetime(2019, 1, 1) end = datetime(2019, 30, 7) ...


Only top voted, non community-wiki answers of a minimum length are eligible