How to find sector/industry and market cap for securities in my portfolio using R?

Question

I am working on a project where I need sector/industry classification and market cap for some securities, many of them are not in SP500, but are part of US market. I am using R for this. I don't have access to any paid data set.

I just started using Quantmod.

I would really appreciate any help.

Answer 1

I have found the IEX API to be extremely valuable. Documentation can be found here: https://iextrading.com/developer/docs/#roadmap

Unfortunately, it looks like they're retiring this API but their new platform appears to still have a free tier that I'm guessing can still provide you with the basic data you're looking for. Here's an example api call for AAPL: https://api.iextrading.com/1.0/stock/aapl/quote

Answer 2

I recently needed market capitalisation data for a large sample of stocks and had no access to paid databases. I therefore resorted to scrape the data from https://finance.yahoo.com using a small webscraping function:

library(tidyverse)
library(rvest)
library(crayon)

scrape_MC <- function(ticker){
  message(blue("Scraping "),
          white(ticker))
  ret <- try(paste0("https://finance.yahoo.com/quote/",
                    ticker) %>%
               read_html()%>%
               html_table() %>%
               reduce(.f=bind_rows) %>%
               as_tibble() %>%
               filter(X1 == "Market Cap") %>%
               pull(X2),
             silent = T)

ifelse(class(ret)=="try-error",
     ticker,
     ret)
}

You can use this function then to scrape individual tickers:

scrape_MC("AAPL")

which returns:

Scraping AAPL
[1] "1.969T"

The data is quoted depending on the unit and sometimes errors are reported, I fixed this using this code snippet:

MC <- MC_data %>%
filter(str_detect(MC, "^\\d")) %>%
  mutate(MC = str_replace(MC, ",", ""),
       Unit = str_extract(MC, "[:alpha:]$"),
       MC = as.numeric(str_remove(MC, "[:alpha:]$")),
       MC = case_when(Unit == "B" ~ MC * 1000,
                       Unit == "M" ~ MC,
                       Unit == "T" ~ MC * 1000*1000,
                       is.na(Unit) ~ MC / (1000*1000)
       )
  ) %>%
  select(-Unit)

You could adapt this to scrape the industry or sector as well.