Questions tagged [data]
Questions about handling, obtaining, generating, or analyzing all types of financial or economic data. Please use a more accurate tag if possible; for instance: tick-data, fundamentals, market-data, option-data, ticker-mapping, etc.
112 questions with no upvoted or accepted answers
10
votes
0
answers
454
views
What is the most convenient data structure for backtesting a model of futures options prices?
I have an empirical model for the dynamics of futures prices in a particular market that I have implemented using a long series of the front five contracts. (I account for the roll in my model.) I ...
- 435
10
votes
0
answers
1k
views
option chain data visualization, sunburst
I think option chains are not represented in the best way. With more and more options products coming out and trading on the various exchanges, I see vendors struggling to keep up with a good way to ...
- 1,872
5
votes
0
answers
103
views
Anyone know if this daily report discontinue to publish? Goldman Sachs - "Global Index Volatility and Correlation Monitor"
I used to receive this daily report in my workplace from Goldman Sachs mailing list but the mailing list discontinued in May 2019 without any notice.
The report is an pdf attachment which send from "...
- 51
4
votes
0
answers
203
views
Match different option high frequency databases
I downloaded the “E-mini S&P 500 (Dollar) Options for 1/10/11” Top-of-Book (BBO) data. If you are interested you may download the data from the following link (approx. 80MB zipped and 1GB unzipped)...
- 111
3
votes
0
answers
189
views
Identifying the same company behind stocks traded in different exchanges (e.g. CUSIP and WKN in 1990ies)
I would like to merge two data sources with companies headquartered in Germany between 1980-2000, but listed in Germany and the US. One source identifies "German companies" by their CUSIP, the other ...
- 31
3
votes
0
answers
666
views
Replacement for Moodys BAA and AAA series
Along with many other people, I have been using Moody's seasoned BAA and AAA corporate bond yield series for my research for some time. I use them primarily to compute and analyze the quality spread. ...
- 141
3
votes
0
answers
81
views
How to get swap reporting data from repositories ICE and CME?
Bloomberg and DTCC both release swap reporting data in form of downloadable files which can be analyzed later on. I have been looking for the same on ICE and CME sites, but couldn't find.
How can I ...
- 133
2
votes
0
answers
88
views
How can you explain the substantially lower trading volume shown in IBKR?
I am using IBKR API to download historical data. The trading volumes are always substantially lower than from Yahoo Finance and Nasdaq.com, often lower by 50%. It is impossible to be explained by ...
- 131
2
votes
0
answers
157
views
Back testing fixed income for bond portfolio management
When doing back test trying to replicate a paper, how do you handle bonds that are maturing in an index?
Say I was trying to create a strategy, how do I account for bonds that mature in terms of ...
2
votes
2
answers
2k
views
Is alpha vantage api for fundamental data reliable?
Can anyone speak to the reliability of the Alpha Vantage (AV) api for fundamental stock data? I have tried for a couple of stocks to get balance sheet data, and it seems close to accurate but I feel ...
- 121
2
votes
0
answers
1k
views
Did AlphaVantage drop the swedish stock exchange?
I have used AlphaVantage in the past to get data for the Stockholm Stock Exchange: I don't remember exactly the symbol I used, but it was something along the line of OMX:ABB or STO:ABB (for, say, ABB)....
- 21
2
votes
0
answers
67
views
Data sources for historical ICE settlement option prices, volume & open interest
I am looking for long history for historical settlement option prices, volume & open interest at ICE Europe (specifically fixed income). This seems to be more challenge than I could imagine. ICE ...
- 151
2
votes
0
answers
171
views
Mapping international firms in I/B/E/S to Compustat
Follow up to my previous question on how to map I/B/E/S to Compustat.
A big chunk of firms in both Compustat and I/B/E/S failed to map via CRSP for me. After some investigation, I noticed that they ...
- 237
2
votes
0
answers
114
views
FTSE 350 sector index historical data
Does anyone know where I can find (at least 10 years worth of) end of day historical (op, hi, lo, cl) data for the FTSE 350 Sector Indices?.
I need the data for ...
- 1,398
2
votes
0
answers
87
views
Is there any research available on profile of the market participants in different markets?
I know markets are by large anonymous, is there any current research or data sources publicly available across different exchanges and/or investment instruments on the profiles of their investors?
I ...
- 207
2
votes
0
answers
164
views
How to reset indicators in quantstrat / quantmod?
I am currently trying to back test a strategy on a file which contains minutely data for the whole month. I would like to reset my indicators at 3:30 PM . Is there a way I can reset the indicators ...
2
votes
0
answers
2k
views
Advice on linking DUNS to GVKEY
I'm working with US Government procurement contracts and I'm like to merge the contract data with stock data from CRSP and firm fundamentals from Compustat.
The contract data uses the Dun & ...
- 141
2
votes
0
answers
273
views
Intraday return and volatility figures some sense check
Some questions about intraday returns and volatility figures. It is with the objective of sense checking.
Firstly, for Security A, I am calculating the five minute interval return numbers over 29500 ...
- 21
2
votes
0
answers
46
views
Feedback on Video Metadata Extraction
I've developed/patented a metadata extraction methodology, which we've used to build a variety of alternative datasets for funds.
One of the more difficult issues in this field, is getting useful ...
- 21
2
votes
0
answers
131
views
Equity Index Announcement Data
Does anyone know of a data source that provides announcements of future index events (adds/drops/re-balances), specifically for equity indexes?
I know these are mostly available somewhere on each ...
- 468
2
votes
0
answers
364
views
Data source for a corporate bond yield curve?
Yield curves are a valuable tool for economic analysis. It is particularly interesting to analyse the difference between Government Debt yields and Corporate Debt yields (credit spreads). This gives ...
- 237
2
votes
0
answers
59
views
Time between end of use of ticker symbol by one company and beginning of use by another?
I have a CRSP stock dataset that goes up to december 2013. I'm trying to append yahoo finance data to it in order to bring it up to the current day. However, there are ticker symbols that once ...
- 249
2
votes
0
answers
106
views
Historical list of Primary Dealers in Europe
I would be interested in a list of all the banks and financial institutions that have been Primary Dealers in the European Union from the 1980s until today. For a current list you can check this pdf ...
- 1,896
2
votes
0
answers
55
views
what kind of test for volatility and where find the data
I am working on a model for stochastic volatility. In short, the model try to capture that the volatility goes up suddenly after a shock (war, policy, financial events, etc) and then goes down slowly, ...
- 21
2
votes
0
answers
735
views
What is the algorithm for the Fama-Bliss instrument selection?
Fama-Bliss discount bonds are defined through a straight forward calculation (Famma, Bliss 1987). For the purposes of a project I need to derive forward rates, yields &c. for periods not included ...
2
votes
0
answers
1k
views
fetch from yahoo! finance database - varying number of ticks
To test a model with real-life data, I used the fetch-function in matlab to connect to the database of yahoo! finance. My code to try and get 7 different assets' returns is the following:
...
- 519
2
votes
0
answers
425
views
Fluid dynamics for order book depth modelling
Would someone be able to give me an idea what type of fluid dynamics I could look at for modelling the order book? My background is more signals-related maths (correlation, covariance, fourier etc).
...
- 610
2
votes
1
answer
160
views
Where to find Investment home bias data & Historical country weight in World MSCI
I need the following data and struggles to find it, maybe some of you can help me. Note: I'm a student and in our university ain't Bloomberg nor Reuters.
Investment home bias, e.g in 2012 US ...
- 257
1
vote
0
answers
18
views
How many orders does a big exchange receive in a trading day?
How many orders does a big exchange receive in one trading day in their most traded asset?
E.g. how many orders does Nasdaq receive for AAPL in one day?
I am interested in a rough estimation and can't ...
- 115
1
vote
1
answer
82
views
Stocks, Bonds, Bills, and Inflation® (SBBI®) Yearbook - replacement
One of the main aggregate datasets for historical returns on different assets classes (the Stocks, Bonds, Bills, and Inflation® (SBBI®) Yearbook) is being discontinued. Source: https://www.kroll.com/...
- 8,621
1
vote
0
answers
54
views
Explanation for a Data from Treasury Direct
I am dealing with a treasury data from TreasuryDirect. Due to the lack of documentation, I am having a hard time understand the meaning of the variables. It would be great help if anyone can share any ...
- 11
1
vote
1
answer
110
views
Sampling dollar bars for ML model of multiple tickers
I have a Neural Network model that provides predictions for the future returns of a portfolio comprising stocks and cryptocurrencies. The original model operates on standard time bars and generates ...
- 213
1
vote
0
answers
99
views
How to aggregate qualitative data?
I am trying to look for methods to aggregate and find the average of qualitative data.
There are 20 qualitative measures, each divided unevenly into 4 cycles labeled 1-4. I am trying to find which ...
- 131
1
vote
0
answers
60
views
ML/DS in fixed income asset management
I am new to the topic but I would like to read papers/books/anything interesting to learn more how ML and data science is used in buy side Fixed income Asset management firms. Factor investing/signals/...
- 493
1
vote
0
answers
74
views
Historical data on valuations for internet companies during dot-com bubble
I am looking for data on historical valuations for internet companies during the years of the dot-com bubble (2000 - 2002). I know that big auditors have or at least have access to such data on ...
- 27.7k
1
vote
0
answers
300
views
Machine Learning model forecasting on real time data in python
I’m building a Forex trading system based on machine learning with Python and brokers API. I get price time series data + fundamental data and then i train the model on that. Model means SVM, RF, ...
1
vote
0
answers
48
views
How to use pivot points for a sell/buy order?
I have implemented some trading strategies like macd and sma. When the lines are crossing they give a sell or buy signal.
Now I have calculated pivot points of one day
...
- 111
1
vote
0
answers
235
views
Literature on realized volatility and sampling frequency?
I was looking for some papers that explicitly show how realized volatility changes as we change the sampling frequency. For example, comparing the annualized volatility estimated from daily data is ...
- 1,910
1
vote
0
answers
50
views
Which strategy did the fund most likely follow?
The following time series represents the return stream of a real hedge fund. Which
strategy did the fund most likely follow?
I did this data superimposition on Matlab. So, there should be some data ...
- 111
1
vote
0
answers
2k
views
Historical data for total market capitalization
I'm working on a data analytics certificate and need historical data on the total market capitalization for the US for a course project. Can anyone suggest a source?
- 11
1
vote
0
answers
75
views
API returning company tickers found in provided news article
I'm looking for a REST API (paid or free) that accepts a string containing a news article (example below) as payload and returns an array of company tickers (eg TSLA for Tesla) mentioned in the ...
- 382
1
vote
0
answers
52
views
Identifying primary share class of U.S. firms and deleting excess observations
I have a fairly large dataset of CRSP firms with their daily closing prices from 2006 till 2017. I need to filter the data so that I have one daily observation per unique firm. I've identified the ...
- 29
1
vote
0
answers
46
views
amount of data - option pricer calibration
A practical question. When you calibrate an option pricer (whatever the model is), do you use data from several days or just one day (last trading day)? I noted some papers use data from one single ...
- 296
1
vote
0
answers
355
views
How to handle database updates for splits/dividends?
I'm finding it an difficult task to maintain a database of stock prices. My main problem is how to efficiently handle splits and dividends. Is it better to handle this in a database with adjusted ...
- 23
1
vote
0
answers
47
views
Transform 24hr cumulative volume to sampled periods
I have a Python Dataframe with cryptocurrency data that has three columns: time, 24hr volume and price.
The time is the time at which the data was received from the exchange, price is the last price ...
- 398
1
vote
0
answers
280
views
Any databases with CEO letters and shareholder letters?
I am researching disclosures on CEO letters and Shareholder letters present in annual reports of listed companies in the London Stock Exchange.
Can someone recommend a database where the annual ...
- 111
1
vote
0
answers
391
views
List of US Industry sector ETFs that map to ICB classification
I am in the prcess of carrying out intra-and intra sector analysis of US stocks, and am proxing sectors with sector ETFs. I therefore, need to catalog the list of US sector ETFs - but have been unable ...
- 1,398
1
vote
0
answers
236
views
Winsorization and Standardization in data manipulation
Suppose there are two time series of data, ask price and bid price in options. They are used to calculate bid-ask spread($\frac{P_A-P_B}{P_{Mid}}$).
I attempt to use winsorization and ...
- 111
1
vote
0
answers
363
views
What symbols to use on Google Finance and Yahoo finance OMX30
I want to download historical data using pandas_datareader for OMX30, however I cant find any symbol which can be found using pandas_datareader.
For example:
STO:...
- 111
1
vote
0
answers
868
views
Commodity index member weights (historical) for S&P GSCI and BCOM
Does anyone know a datasource for retrieving the historical index member weights for the two major commodity indices S&P GSCI and Bloomberg Commodity Index?
My institution provides access to ...
- 111