What is the possible source of this filing_index.csv?

Question

I am reading the book Machine Learning for Algorithmic Trading by Stefan Jansen. And there is a file called filing_index.csv, which contains index information of some SEC filings. It can be accessed via the following link.

https://github.com/stefan-jansen/machine-learning-for-trading/blob/main/data/sec-filings/filing_index.csv

Here are a few rows from the beginning of the file

"CIK","COMPANY_NAME","FORM_TYPE","DATE_FILED","EDGAR_LINK","QUARTER","TICKER","SIC","EXCHANGE","HITS","YEAR"
1000180,"SANDISK CORP","10-K","2013-02-19","edgar/data/1000180/0001000180-13-000009.txt",1,"SNDK","3572","NASDAQ","3",2013
1000209,"MEDALLION FINANCIAL CORP","10-K","2013-03-13","edgar/data/1000209/0001193125-13-103504.txt",1,"TAXI","6199","NASDAQ","0",2013
1000228,"HENRY SCHEIN INC","10-K","2013-02-13","edgar/data/1000228/0001000228-13-000010.txt",1,"HSIC","5047","NASDAQ","3",2013
1000229,"CORE LABORATORIES N V","10-K","2013-02-19","edgar/data/1000229/0001000229-13-000009.txt",1,"CLB","1389","NYSE","2",2013
1000232,"KENTUCKY BANCSHARES INC  KY ","10-K","2013-03-28","edgar/data/1000232/0001104659-13-025094.txt",1,"KTYB","6022","OTC","0",2013
1000298,"IMPAC MORTGAGE HOLDINGS INC","10-K","2013-03-12","edgar/data/1000298/0001047469-13-002555.txt",1,"IMH","6798","NYSE MKT","0",2013

This book doesn't seem to have mentioned the source of this filing index file. As someone who is unfamiliar with SEC filings, could anyone please help me find out the source of this file? Or is it made by the book author himself?

Answer 1

This is scraped from the Edgar database using a Python package or his own web-scraping tools:

I believe the author constructed the filing_index.csv using a similar script/package as the Python package called python-edgar (see here for documentation). The package constructs a master-index file with all available information from the Edgar database since a user-defined year (pre-defined to 1993).

As is also expressed in the documentation, you can then get company filings via the master-index file by filtering for eg. company (CIK number) and form-type using grep in Python (see example from the Github documentation). For completeness, I've provided quote-snippets from the package documentation that details how to get the needed information:

Stitch quarterly files to a master file

python-edgar does only one thing and does it well: getting and cleaning uncompressed quarterly index files to your computer. Use command line tools, in the spirit of unix philosophy, to stitch these index files together and create our master index file. [...]

Grab filings from a specific company

Now that we have downloaded the index files it becomes easy, with a bit of command line scripting, to quickly filter by company and extract URLs to the filings we want with grep. In the following example we grep by CIK (1000045), store the output in an intermediate text file, which we re-open with cat and grep again by form 10-K. Prefix the paths with https://www.sec.gov/Archives/ and you'll get the full URL.

The output from the above example, looks very similar to the information in the filing_index.csv with less details than the authors csv-file (missing columns beyond "EDGAR_LINK"):

1000045|NICHOLAS FINANCIAL INC|10-K|2015-06-15|edgar/data/1000045/0001193125-15- 
223218.txt|edgar/data/1000045/0001193125-15-223218-index.html
1000045|NICHOLAS FINANCIAL INC|10-K|2016-06-14|edgar/data/1000045/0001193125-16- 
620952.txt|edgar/data/1000045/0001193125-16-620952-index.html
1000045|NICHOLAS FINANCIAL INC|10-K|2017-06-14|edgar/data/1000045/0001193125-17- 
203193.txt|edgar/data/1000045/0001193125-17-203193-index.html
1000045|NICHOLAS FINANCIAL INC|10-K|2018-06-27|edgar/data/1000045/0001193125-18- 
205637.txt|edgar/data/1000045/0001193125-18-205637-index.html

The author could then convert the above output to a csv-file and possibly merge it with securities data from the CRSP database (or similar), in order to give you the last columns missing in the output.

---

In conclusion, I do not believe you will find any easy way to download the same type of information as provided in filing_index.csv. This might also be the case why the author has not mentioned the source. Nevertheless, I hope this provide some insight.