109
$\begingroup$

I have a very basic data question: how to get a list of all common stocks traded on NYSE, NASDAQ and AMEX? I would need to be able to get the approximate list of common stocks as is available in Telechart... I can not get this data at eoddata, for example...

I would like to calculate market breadth indicators and would like to find out how many of common stocks traded were up or down 4% (breakouts/breakdowns) (Cl-Lag(Cl) > 0.04 (4% breakout), Cl-Lag(Cl) < -0.04 (4% breakdown)), how many of common stocks traded are down/up 25% in quarter etc.

My first problem is how to get a list of symbols with only common stocks (no ETFs).

Improve this question
$\endgroup$
7
  • $\begingroup$ Hi Samo, welcome to quant.SE and thanks for your question. Please consider registering in order to edit and respond to answers and to gain further privileges. $\endgroup$
    – Tal Fishman
    Commented Aug 10, 2011 at 1:36
  • $\begingroup$ nasdaq site is extremely dated. bloomberg is up to date. $\endgroup$
    – DroBuddy
    Commented Jan 22, 2014 at 2:13
  • 3
    $\begingroup$ NASDAQ site has been updating this entire time. sure the site design is old, but the data is good. $\endgroup$
    – Adam
    Commented Feb 5, 2019 at 15:01
  • 8
    $\begingroup$ In 2019, the data is here: ftp.nasdaqtrader.com/symboldirectory $\endgroup$
    – duhaime
    Commented Mar 14, 2019 at 16:34
  • $\begingroup$ Try this API endpoint to lookup tickers rapidapi.com/logicione/api/… and this endpoint rapidapi.com/logicione/api/… to lookup/check S&P500 traded (i.e. actively traded) companies. You can try the API for 50 req/day free. For more calls pay a small amount. $\endgroup$
    – Watt
    Commented Aug 1, 2020 at 7:53

7 Answers 7

Reset to default
122
$\begingroup$

NASDAQ makes this information available via FTP and they update it every night. Log into ftp.nasdaqtrader.com anonymously. Look in the directory SymbolDirectory. You'll notice two files: nasdaqlisted.txt and otherlisted.txt. These two files will give you the entire list of tradeable symbols, where they are listed, their name/description, and an indicator as to whether they are an ETF.

Given this list, which you can pull each night, you can then query Yahoo to obtain the necessary data to calculate your statistics.

UPDATE: More information about these files and their fields can be found here.

Improve this answer
$\endgroup$
10
  • 3
    $\begingroup$ Very nice! I'll have to add a function to TTR to access these files. $\endgroup$
    – Joshua Ulrich
    Commented Sep 8, 2011 at 18:30
  • $\begingroup$ Does Yahoo allow you to run a script to do thousands of HTML fetches against their servers? I know some web sites will ban your IP if you try that. $\endgroup$
    – pacoverflow
    Commented Jun 15, 2013 at 7:42
  • $\begingroup$ @pacoverflow They do, at the very least, allow ~7500 sequential individual requests. It is also possible to download data for up to 200 symbols from Yahoo! in one request. $\endgroup$
    – Eliezer P
    Commented Jan 26, 2014 at 3:14
  • 8
    $\begingroup$ ftp.nasdaqtrader.com/SymbolDirectory ftp.nasdaqtrader.com/SymbolDirectory/nasdaqlisted.txt ftp.nasdaqtrader.com/SymbolDirectory/otherlisted.txt $\endgroup$
    – 1mike12
    Commented Jul 3, 2017 at 15:08
  • 5
    $\begingroup$ I created an account on this site just to upvote this answer (and the question). I can't believe how hard this type of information is to find, reliably. $\endgroup$
    – user42717
    Commented Sep 23, 2019 at 16:02
32
$\begingroup$

Slightly ugly bash one-liner for a sorted JSON array:

echo "[\"$(echo -n "$(echo -en "$(curl -s --compressed 'ftp://ftp.nasdaqtrader.com/SymbolDirectory/nasdaqlisted.txt' | tail -n+2 | head -n-1 | perl -pe 's/ //g' | tr '|' ' ' | awk '{printf $1" "} {print $4}')\n$(curl -s --compressed 'ftp://ftp.nasdaqtrader.com/SymbolDirectory/otherlisted.txt' | tail -n+2 | head -n-1 | perl -pe 's/ //g' | tr '|' ' ' | awk '{printf $1" "} {print $7}')" | grep -v 'Y$' | awk '{print $1}' | grep -v '[^a-zA-Z]' | sort)" | perl -pe 's/\n/","/g')\"]"

EDIT: Filtered out test stocks

Improve this answer
$\endgroup$
4
  • 14
    $\begingroup$ I just had a nerdgasm $\endgroup$
    – airstrike
    Commented Apr 4, 2016 at 1:19
  • 2
    $\begingroup$ +2 for taking out the Test stock listings. $\endgroup$
    – DragonLord
    Commented Aug 11, 2020 at 18:35
  • $\begingroup$ Got this: head: illegal line count -- -1 head: illegal line count -- -1 ["-n ",""] $\endgroup$
    – Elimination
    Commented Jan 29, 2021 at 14:04
  • $\begingroup$ @Elimination I think you have to put a space between the -n -1 arguments for non-GNU utilities. On my Mac, I got it to work using ghead, gmail, gawk, etc. from Macports. $\endgroup$
    – Eric Brown
    Commented Feb 6, 2021 at 12:59
20
$\begingroup$

Bloomberg Open Symbology has this list. Look in the Common Stock precanned file. This will have a bit more data than you probably need as it has a separate entry and unique id for each place an equity is traded. However it is probably the highest quality list available for free anywhere.

As for filtering ETFs are broken out in a separate file (Equity_ETP) so you simply need to filter by exchange.

Update November 2017: BSYM is no more and its replacement does not have prebuilt files as far as I can tell.

Improve this answer
$\endgroup$
7
  • 1
    $\begingroup$ This is great because it has all securities globally - I'd found it difficult to get non-US symbols before this. $\endgroup$
    – fantabolous
    Commented Jul 7, 2014 at 5:21
  • $\begingroup$ @fantabolous Can you provide an example for how to use this to get all tickers? Can't figure out how to use it. $\endgroup$
    – mjs
    Commented Sep 23, 2014 at 14:01
  • $\begingroup$ @Hamidam E.g. if you want all HK common stock, from the left panel of the link above choose Market Sector: Equity, Sec Type: Common Stock, Country: Hong Kong. Click Submit. Then click "Excel" in top right corner. Done. $\endgroup$
    – fantabolous
    Commented Sep 24, 2014 at 6:30
  • $\begingroup$ Also, for equities if you select a country (not exchange) and then deselect All Exchanges it'll give you only the primary exchange. $\endgroup$
    – fantabolous
    Commented Sep 24, 2014 at 7:23
  • 5
    $\begingroup$ Not working.... $\endgroup$
    – Brethlosze
    Commented Nov 10, 2017 at 14:11
18
$\begingroup$

You can download all stocks on the three exchanges listed in your question from the NASDAQ website: http://www.nasdaq.com/screening/company-list.aspx.

It looks like removing those entries with an industry of "N/A" will eliminate ETFs and other funds from the list.

Improve this answer
$\endgroup$
3
9
$\begingroup$

The stockSymbols function in the R package TTR pulls the data from nasdaq.com that @bellamyj mentioned. It also attempts to convert the symbols to a format acceptable to Yahoo Finance.

That said, I'm not certain how to filter this list for only common stocks. There are 1275 securities with "n/a" Sector or Industry, leaving ~5000. Perhaps the remaining categories in Sectors and Industries can help you further refine the list.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ I have a problem using this function, in fact I only get gibberish. I posted on [R-Sig-Finance] on the matter. $\endgroup$
    – Owe Jessen
    Commented Aug 12, 2011 at 13:12
  • 2
    $\begingroup$ @Owe: I answered on R-SIG-Finance. Basically, the file changed and I haven't pushed the update to CRAN yet. The R-Forge version of TTR works. Sorry for the confusion. $\endgroup$
    – Joshua Ulrich
    Commented Aug 12, 2011 at 13:18
  • 1
    $\begingroup$ No problem, thanks for the fast reply and for writing this function. $\endgroup$
    – Owe Jessen
    Commented Aug 12, 2011 at 13:27
3
$\begingroup$

I've been using this API called Stock Coverage from unibit. It covers around 70,000 stocks from 49 exchanges. Though I'm not sure if it is the most updated. A few recent IPOs can be found in the list.

The ticker standard seem to complies with Yahoo's and it works with many of the other APIs in their site.

All their listed tickers can be found on this public domain.

Improve this answer
$\endgroup$
0
$\begingroup$

Do you have an existing brokerage account or perhaps a data provider you're subscribed to? I was able to previously receive a complete list of stocks for the full Equities universe on all the US exchanges from algoseek.com

Improve this answer
$\endgroup$
1

Not the answer you're looking for? Browse other questions tagged or ask your own question.