Questions tagged [data]
Questions about handling, obtaining, generating, or analyzing all types of financial or economic data. Please use a more accurate tag if possible; for instance: tick-data, fundamentals, market-data, option-data, ticker-mapping, etc.
500 questions
3
votes
1
answer
602
views
API for Commodities Stock markets
I would like to have a clear picture about lithium investments all over the World.
I like this website where I can see all companies related to lithium on the Australian Market.
The website also ...
1
vote
1
answer
910
views
Which spread to use to analyse CDS data from Markit
So I am currently doing some analysis on CDS Data and I am using Markit to extract this data. However, I am a bit confused regarding the quotation standards here. I want to investigate the ...
- 11
-1
votes
1
answer
85
views
Can NumPy calculate the % change the way it is shown in multiple instrument charts?
I have closing prices for multiple equities in NumPy arrays (or a pandas timeseries DataFrame).
I like to calculate the % change numbers for the closing prices in the DataFrame the way it is shown in ...
- 119
0
votes
0
answers
70
views
Should I always round the data before even trying defining and backtesting my trading strategy?
Newbie here, it happens that I have 8 months of OHLC price data set at 1 hour timeframe from a particular cryptocurrency (ticket) called ALICE, here's a little ...
- 23
0
votes
0
answers
54
views
SONIA Reference Data Calculation / OIS Data
I am reading material reference rates calculation for SONIA by refinitiv. It says the data is calculated based on OIS Rates between 10:50AM and 11:00 AM.
https://www.refinitiv.com/content/dam/...
- 11
0
votes
0
answers
64
views
Italy Zero Coupon Yields
I am looking for historical data for Treasury bills and bond yields for Italy on a monthly or daily basis. Where can I get this data?
- 1
2
votes
0
answers
157
views
Back testing fixed income for bond portfolio management
When doing back test trying to replicate a paper, how do you handle bonds that are maturing in an index?
Say I was trying to create a strategy, how do I account for bonds that mature in terms of ...
1
vote
1
answer
379
views
How do you identify top shareholders in SEC filings?
Schedule 13D is filed when a shareholder crosses 5% ownership and 13F is filed by fund managers to list their portfolio holdings. These are the only filings that I am currently aware of related to ...
- 131
0
votes
0
answers
151
views
How can I calculate the annual Standard Deviation for Sharpe Ratio of Daily Portfolio Returns?
I'm somewhat confused with regards to calculating the annual standard deviation and Sharpe ratio for my portfolio of daily returns.
I have daily data ranging from 1960-2020 and use Excel to make some ...
- 1
1
vote
2
answers
377
views
Where do big data companies get data?
First time posting at Quantitative Finance, hope the question is well suited in this stack section.
I was wondering, where does big companies like Bloomberg or Reuters get their data?
Since now, in my ...
- 13
0
votes
2
answers
277
views
Where can I get fundamental data on US public stocks in original form?
I'm looking for fundamental data on US public companies. If anyone could recommend paid or free sources of the fundamentals in original, non-standardized form that would be great. I tried scraping the ...
1
vote
2
answers
750
views
What kind of data cleansing/scrubbing are hedge funds doing?
It's a well-known fact that several hedge funds have a handful of PhDs just doing data cleansing. All day. Every day.
What kind of data cleansing are they actually doing? Is it really that difficult? ...
- 181
4
votes
1
answer
227
views
Help with pulling company fundamentals from EDGAR
I need help understanding why there are many different values for a specific metric reported by a company's 10-K or 10-Q in the EDGAR database.
I've downloaded the entire US equity universe in JSON ...
- 41
0
votes
0
answers
1k
views
Fama-French factors creation using Bloomberg data
The standard FF factors as published by K. French (https://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html) are created using CRSP data. If one downloads the (US) factors, the first ...
- 273
12
votes
1
answer
3k
views
What's the most efficient way to store options and time series data for backtesting?
I would like to know what database would you guys use for storing around 500GB-1TB of options and time series tick data.
The idea is to use it for backtesting so it would have to be as efficient as ...
- 223
2
votes
1
answer
2k
views
Bloomberg API / Excel Add In - Delisted Stocks
So I am using the Bloomberg Excel add in but can also use the API. My dilemma is I have a list of tickers containing SP500 historical constituents and I have retrieved 600 of them from an alternative ...
- 23
0
votes
0
answers
322
views
Historical ISM report on business?
Is there any place to download historical ISM ROB (report on business)? The website only shows current and previous month. ISM ROB
- 63
6
votes
2
answers
842
views
Algorithm / source to calculate historical expiry dates of futures
I can find several source on this site where to find expiry dates of coming futures contracts.
I am looking for a (e.g. Python) algorithm or a data source where I can find historical dates when a ...
- 61
0
votes
1
answer
137
views
How could I identify peers of companies (stocks) at scale?
Context:
I'm making a small script to screen through thousands of companies at scale. A feature that I want to include would be relative metrics, e.g. is this company growing faster than its peers, ...
- 101
1
vote
0
answers
26
views
Where to find a list of all the operable exploration / mining companies conducting operations in Canada [closed]
Looking to create a database firstly of all the exploration / mining companies conducting research in Canada. TSX and TSX-V have very nice spreadsheets with several filtering operations, which is very ...
- 11
1
vote
2
answers
2k
views
When looking for arbitrage among a LARGE amount of assets, is there an optimal way?
Looking for arbitrage opportunities when looking at 3 pairs of related currencies is easy. However if we assume that we have a large amount of currencies, is there an optimal way to swipe through them ...
- 135
2
votes
2
answers
2k
views
Is alpha vantage api for fundamental data reliable?
Can anyone speak to the reliability of the Alpha Vantage (AV) api for fundamental stock data? I have tried for a couple of stocks to get balance sheet data, and it seems close to accurate but I feel ...
- 121
0
votes
1
answer
165
views
Non-fixed stationary "conversion"
Dear users of StackExchange,
I was wondering why the log returns of a fixed period of time is such a common use in "transforming" a time series into a more stationary one?
I thought that ...
- 25
1
vote
1
answer
77
views
Historical Economic Release Calendar
Does anyone know if any providers have a historical economic release calendar? For example, releases of things like ADP private employment, or ISM surveys? I ask because I would like to backtest a ...
- 469
11
votes
3
answers
9k
views
Red Black Trees for Limit Order Book
Why do people suggest using red black trees/balanced binary trees for the levels in a limit order book?
Why are they algorithmically ideal?
- 2,662
2
votes
2
answers
1k
views
What are the proper ways to do order book downsampling?
I have an access to the order book dataset, which was sampled with resolution that is too high for my sandbox experiments with it. Because of that, I was wondering, what would be the correct way to ...
1
vote
3
answers
225
views
OHLC Data for US Govt. Treasury Yields?
Is there open-high-low-close data available at the daily timescale for yields? The data I can find from the Federal Reserve reports only one price, and moreover, I'm not sure when the data is reported....
- 469
-2
votes
1
answer
201
views
What are good examples of using Big Data for a trading strategy?
As the title say, I am looking for trading strategies in the stock, bond, forex market, that rely on the usage of Big Data. The more out of the box strategy the better. Also if anyone knows any ...
- 8,621
0
votes
2
answers
1k
views
Large data of Equity Price download from Eikon thomson reuters datastream
I like to download all Chinese ( more than 4000 stocks) stocks daily price data from 1990 to 2020 from Eikon Thomson Reuters datastream. However, I failed to select and download all stocks together. ...
1
vote
2
answers
238
views
Alternative data providers
What data vendors/providers have you come across or used for alternative datasets? Not talking about market data here. Things like rainfall, coronavirus stats, new car production in China etc. I
1
vote
1
answer
76
views
should I persist the daily/weekly bar data
Suppose a small quant group (4 guys) and the tick data is in hand, shall they persist the historical daily/weekly or even smaller scale like hourly/minutes bars data, or just do the on-demand ...
- 11
1
vote
2
answers
632
views
Storing options EOD time series in Flat Files
I have purchased data for EOD settlements of options prices for USA futures for personal use. I will not need multiple user access or real time access. I am not an expert programmer but use C# and R ...
- 11
0
votes
1
answer
226
views
Swap data- couldn't find any
I'm a student and i amm looking for a swaps rates historical data for long tenors in purpose to estimate yield curve (for example in GBP). My question is where could I find it?
- 1
2
votes
1
answer
621
views
Historical intraday dataset with penny stocks with a gap-up of 10% or above
I am trying to find intraday datasets of pennystocks. My criteria for the dataset is that it needs to only contain pennystocks where the gap-up was 10% or above.
I am looking for a free option but ...
- 133
1
vote
1
answer
313
views
Best way to treat negative P/E and extremely high P/S in data?
I'm working on a data that deals with start-up/similar companies data. A lot of the companies have negative P/E and/or little to no sales. Is there good ways to create meaningful data (like averages, ...
- 13
0
votes
0
answers
305
views
How to download all tickers listed in the Tokyo stock exchange?
I'm trying to download all tickers listed in the Tokyo stock exchange (according to their website, there should be 3,756 tickers in total).
What I have tried so far:
I have tried the Yahoo-ticker-...
- 101
2
votes
5
answers
657
views
Top 500 Companies in the World – Data
At the basis of an algorithm, I need an Excel/CSV file containing the world's top 500 companies (with respect to market cap, preferably free-float if possible) by Name, Country, Market-Cap (in USD, or ...
- 31
1
vote
0
answers
74
views
Historical data on valuations for internet companies during dot-com bubble
I am looking for data on historical valuations for internet companies during the years of the dot-com bubble (2000 - 2002). I know that big auditors have or at least have access to such data on ...
- 27.7k
0
votes
1
answer
272
views
Cap/Floor on a SpreadOption grid
I have a spread option data from a broker. The rows are the following :
STK
ATM
-0.5
-0.25
... and the values are forward price ( the strikes used are absolute strike and the value of the raw STK is ...
- 11
0
votes
3
answers
134
views
best snp500 trade data
where is the best place to get spx data, all the trades with exact millisecond timestamps a few weeks back? I don't mind paying for such a service
- 101
0
votes
1
answer
414
views
Free website to obtain corporate bond's data similar to cbonds?
Does anyone know of any website where I can obtain corporate bond's data in the same fashion as "cbonds" (they let you look up by issuer, maturity, etc.). Ideally free?
I need to know bond ...
- 326
0
votes
0
answers
534
views
IB API - get data on position that close
I am trying to implement a few new features for my trading system. My plan for now is to initially buy the stocks i wants (using Vwap) and when the order has closed (bought all the quantity) I want to ...
1
vote
0
answers
300
views
Machine Learning model forecasting on real time data in python
I’m building a Forex trading system based on machine learning with Python and brokers API. I get price time series data + fundamental data and then i train the model on that. Model means SVM, RF, ...
2
votes
2
answers
143
views
List of public US (NYSE, Nasdaq) companies that filed for bankruptcy in the last 10 years
I am working on a Data Science project on bankruptcy prediction. I am looking for a list of listed US companies that filed for bankruptcy in the last 10 years but I am really struggling to find any ...
- 21
1
vote
0
answers
48
views
How to use pivot points for a sell/buy order?
I have implemented some trading strategies like macd and sma. When the lines are crossing they give a sell or buy signal.
Now I have calculated pivot points of one day
...
- 111
0
votes
0
answers
55
views
transforming variables
I am would like to create a regression model with different variables however before using these variables in my regression model I would like to transform the variable in order to make it more ...
- 339
0
votes
1
answer
413
views
Are there any public LIBOR Data sources?
I am looking for 1 and 3 month LIBOR rates available through an API. I am familiar with the rates available on FRED, but they are only available with a 1 week lag. Unfortunately, I'm looking for no ...
- 1
1
vote
1
answer
321
views
Any database with all of companies previous CEO's?
I'm doing some research on CEO's and company chairmen and I'm looking for a database that contains this data for all the public US companies. The data needs dates associated with the names as well. I'...
- 63
-3
votes
1
answer
63
views
Why no stock's amount data on us stock market?
In chinese stock market , every stock's traded volume and amount (price * volume) data was disclosed.
Why in us stock market no amount data disclosed?
There must be such 7 fields as ...
- 105
0
votes
1
answer
181
views
How to deal with missing stock returns?
If I want to calculate the Covariance between two stocks but there are missing days in both, how can I deal with missing data? I want to use Pairwise deletion and only use the days of which both ...
