1
$\begingroup$

I am working on a project where I need sector/industry classification and market cap for some securities, many of them are not in SP500, but are part of US market. I am using R for this. I don't have access to any paid data set.

I just started using Quantmod.

I would really appreciate any help.

$\endgroup$
1
$\begingroup$

I have found the IEX API to be extremely valuable. Documentation can be found here: https://iextrading.com/developer/docs/#roadmap

Unfortunately, it looks like they're retiring this API but their new platform appears to still have a free tier that I'm guessing can still provide you with the basic data you're looking for. Here's an example api call for AAPL: https://api.iextrading.com/1.0/stock/aapl/quote

$\endgroup$
1
$\begingroup$

I recently needed market capitalisation data for a large sample of stocks and had no access to paid databases. I therefore resorted to scrape the data from https://finance.yahoo.com using a small webscraping function:

library(tidyverse)
library(rvest)
library(crayon)

scrape_MC <- function(ticker){
  message(blue("Scraping "),
          white(ticker))
  ret <- try(paste0("https://finance.yahoo.com/quote/",
                    ticker) %>%
               read_html()%>%
               html_table() %>%
               reduce(.f=bind_rows) %>%
               as_tibble() %>%
               filter(X1 == "Market Cap") %>%
               pull(X2),
             silent = T)

ifelse(class(ret)=="try-error",
     ticker,
     ret)
}

You can use this function then to scrape individual tickers:

scrape_MC("AAPL")

which returns:

Scraping AAPL
[1] "1.969T"

The data is quoted depending on the unit and sometimes errors are reported, I fixed this using this code snippet:

MC <- MC_data %>%
filter(str_detect(MC, "^\\d")) %>%
  mutate(MC = str_replace(MC, ",", ""),
       Unit = str_extract(MC, "[:alpha:]$"),
       MC = as.numeric(str_remove(MC, "[:alpha:]$")),
       MC = case_when(Unit == "B" ~ MC * 1000,
                       Unit == "M" ~ MC,
                       Unit == "T" ~ MC * 1000*1000,
                       is.na(Unit) ~ MC / (1000*1000)
       )
  ) %>%
  select(-Unit)

You could adapt this to scrape the industry or sector as well.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.