1
$\begingroup$

I have a large dataset (taken from Kaggle: https://www.kaggle.com/borismarjanovic/price-volume-data-for-all-us-stocks-etfs/), and I would like to fill in the missing data.

To do that I can (1) iterate through all the data and get a set of dates that at least one stock was traded on, or (2) get a reference list with dates that trading occurred.

I would like to know from where I can get the data for option (2). Any help?

New contributor
Natan ZB is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
$\endgroup$
3
$\begingroup$

QuantLib provides calendars for given countries and exchanges, see here.

Dates can then be intersected using something like NumPy's np.intersect1d, for example numpy.intersect1d(cal_dates, numpy.array(db_etfs.loc[:, 'date'])).

$\endgroup$
  • $\begingroup$ Will check, thanks! $\endgroup$ – Natan ZB Feb 11 at 12:02
1
$\begingroup$

@AlexAbrahams's recommended resource is a decent one but here's another (I think better) approach which solves both (1) and (2):

#!/usr/bin/env python
# Call from the `Data` directory

import glob
import pandas as pd
import datetime


# Get all traded dates, with possible repetition
dates = []
for f in glob.glob('*/*.us.txt'):
    dates += df['Date'].tolist()

# Remove repetitions
dates = set(dates)

# Optional: Convert to `datetime`
dates = map(lambda s: datetime.datetime.strptime(s, '%Y-%m-%d').date(),
            dates)

# Optional: Sort
dates = sorted(dates)

Two reasons to consider doing it this way instead:

  1. You don't need an external, and very large, dependency on QuantLib.

  2. In production, there may be practical, structural or systematic reasons why you can't trade on the days that are in Quantlib but not in your data. For example, the network that your servers are on may be down, causing you to lose data on those dates. Sometimes this loss is connected with events of significant volatility, e.g. a circuit breaker tripping on the exchange causing a glitch in your own software. It doesn't make sense to impute data on the trading schedule "as though" you would've been able to trade on those dates because there's a very repeatable reason why you wouldn't have been able to. Your own data captures these nuances the best as opposed to a third party library.

$\endgroup$
  • $\begingroup$ Hey @madilyn, this is actually option (1); maybe I wrote it in a confusing way... Thanks anyways! $\endgroup$ – Natan ZB Feb 12 at 7:01

Your Answer

Natan ZB is a new contributor. Be nice, and check out our Code of Conduct.

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.