0
$\begingroup$

I would've guessed this question would have been answered somewhere on here before, but I have been going down rabbit holes for days trying to figure it out with no luck...

I want to backtest different value investing strategies, and I'm trying to find a high quality source for historical ticker symbols to create a tradable universe from.

For example, if I wanted to test the performance of a strategy that buys the lowest P/E stocks on Jan 1st and rebalances annually from 2010-2020, I would need to know the universe of ticker symbols available to purchase on Jan 1st for each year. Then, I would have to remove all the tickers that are not relevant like ETFs, warrants, preferred shares, SPACs, etc.

The only data provider I have been able to find that allows you to travel back in time and get retroactive ticker data is Alpha Vantage under their 'Listing and Delisting Status' endpoint. It lets you enter a date as a URL parameter and pulls the active tickers listed on various exchanges at that point in time.

However, the data is a mess. When you pull the data from that endpoint it lists essentially everything as having an asset type of 'stock' or 'ETF'. I have tried running the tickers it outputs through other endpoints like 'company overview' that I thought might classify the asset type/industry of each ticker more accurately but have been unsuccessful.

So for now, I'm stuck with trying to filter everything by the company name. Like if it has 'Warrant' in the name I exclude it, etc. But writing the logic to do that correctly for >10k tickers is basically impossible.

Has anyone found a good solution for this? Thanks in advance!!!

$\endgroup$
0
$\begingroup$

In absence of reference data, one can build the list of tradeable stocks by running a query on end-of-day data and selecting symbols with at least one trade during the given day, for example on the first trading day/week/month of each year in the backtesting period.

In SQL syntax, this would look as follows:

SELECT symbol, date_format(time, 'yyyy') AS dt
FROM atsd_session_summary
WHERE exchange = 'SIP' AND class = 'SIP' AND symbol LIKE 'A%'
  AND date_format(time, 'w') <= 2 AND datetime >= '2020'
GROUP BY exchange, class, symbol, PERIOD(1 year)
ORDER BY symbol, dt

If you have reference data nearby, filtering by security type is another filter one can use. The good news is that security type doesn't change so we can safely apply the current type to historical records. Security type is typically available in all retail data platform APIs, whether Alpha Vantage or others.

  AND tags.entity.type = 'CS'

enter image description here

$\endgroup$
3
  • $\begingroup$ Thanks. I'm trying to use IEX Cloud which has reference data and historical prices. However, the reference data does not include delisted securities BUT their pricing data does include delisted securities. The problem with querying all end of day data is I have to type in a list of symbols I want to retrieve in the path. So it seems I have no way of getting the delisted symbols I need to type in... Is this type of research not possible with free/cheap data providers? $\endgroup$
    – Kyle M
    Apr 6 at 17:54
  • $\begingroup$ IEX Cloud is great, but it's indeed lacking an endpoint that provides a list of traded securities on a given date. You can use Polylgon.io grouped EOD endpoint as a workaround: polygon.io/docs/… Example: api.polygon.io/v2/aggs/grouped/locale/us/market/stocks/… $\endgroup$ Apr 6 at 20:49
  • $\begingroup$ Thank you! Polygon is working great. Appreciate the help. $\endgroup$
    – Kyle M
    Apr 7 at 21:06
0
$\begingroup$

To do this correctly, you need several things:

  • survivorship bias-free price data (that is, data with active and delisted tickers). If you plan to use fundamental data in your investment rules, you also need survivorship bias-free fundamental data.
  • securities master data that is sufficiently rich to allow you to distinguish between different security types (preferred shares, common stock, ADRs, etc.)
  • a way to dynamically filter the universe of securities according to your criteria, based on the subset of securities that were active at the time.

QuantRocket supports this use case well. It includes survivorship bias-free price data for US equities and integrates survivorship bias-free fundamental data from Sharadar. It has rich securities master data. And the Pipeline API is specifically designed for filtering and performing computations on large universes of securities.

Disclaimer: I'm affiliated with QuantRocket.

$\endgroup$
0
$\begingroup$

algoseek.com provides Equity Security Master File which tracks any changes for a security (Ticker, Name, Primary Exchange, etc) and goes back to Jan 2007.

You can filter tickers by SecurityDescription column and a relevant date range to get your target Universe. Also, they provide FIGI and ISIN within it, so cross-referencing with other vendors should be pretty easy.

You can find it available with relevant documentation and data sample at the products page.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.