Questions tagged [database]
A structured collection of data, with meta-data describing is content and meaning. Also for database management systems (DBMS) which are software products for managing and manipulating that data.
67
questions
2
votes
2
answers
97
views
Effective way to persist strategy real time orders to database? [closed]
Background: I'm building a trading system for the crypto market with Python, and currently having problems on how to effectively save my real time orders/trades to disk, so that I could monitor more ...
- 430
0
votes
1
answer
34
views
How to create Industry Portfolios per country? [duplicate]
Regarding the industry portfolio as created by Kennth R. French like here: https://mba.tuck.dartmouth.edu/pages/faculty/ken.french/Data_Library/det_17_ind_port.html I was wondering how I could do this ...
- 1
4
votes
1
answer
1k
views
Efficient way to store orderbook in Python
I am using the Coinbase WebSocket API to extract real-time data about the orderbook for BTC-USD.
I am using the following code to store the snapshots of bids and asks and the changes to the orderbook ...
- 203
0
votes
0
answers
42
views
Data structure to describe ownership of business entities: What would make a good one?
I am building a database of news outlets, and I would like to include information about the ownership of the news outlets. In order to do this to my satisfaction, I would like to store information ...
- 101
1
vote
1
answer
181
views
Where to find dissertations in risk management
I'm looking for open databases of master's dissertations/theses in risk management & quantitative finance written by risk practitioners.
The goal is to find current research topics or problems in ...
- 386
4
votes
2
answers
187
views
Backtesting vs live trading data handling and abstraction
I'm an individual trying to build a trading system which will ideally be eventually scalable to 1-15 second resolution intraday trading strategies. I'm having some trouble understanding the difference ...
- 143
0
votes
0
answers
241
views
Time series data for probability of default (or credit ratings)
I'm currently investigating potential correlations among ESG ratings and credit ratings; more in particular, i'm trying to understand whether such correlation evolved during the last 20 (?) years, and ...
0
votes
3
answers
167
views
database for economic & finance timeseries
I am looking for a technical solution to store economic and financial timeseries (nothing intraday for now, just daily/weekly/yearly)
Most timeseries database I find do not seem to take into account ...
- 2,003
0
votes
0
answers
85
views
Why do researchers exclude firms with total asset less than £1 million?
Chaney, Jeter, and Shivakumar (2004, page 58) exclude firms with less than 1 million pound in total assets
Further, we exclude firms with less than £1 million in total assets
I am wondering why they ...
- 297
1
vote
0
answers
176
views
Are there any best practices for designing high frequency trading systems?
I spent some time trying to design some parts of the system, going over the information I found.
At the top-level, the system looks like this
A "real-time" module that receives market data, ...
- 11
1
vote
0
answers
82
views
Which Times Series Database framework for Python is best for portfolio optimization project?
I am starting to build a portfolio optimization algorithm in Python and want to structure a database to manipulate financial data. Although I have Python experience, I have never used SQL or such ...
1
vote
0
answers
33
views
From SEC Comprehensive Data Set to Clean Balance Sheets/Income Statements
I am looking for advice on how to smartly get from unstructured financial data to a clean summary of balance sheets. I think of this not as a coding problem but of a question how to approach the task -...
- 11
12
votes
1
answer
2k
views
What's the most efficient way to store options and time series data for backtesting?
I would like to know what database would you guys use for storing around 500GB-1TB of options and time series tick data.
The idea is to use it for backtesting so it would have to be as efficient as ...
- 223
1
vote
0
answers
24
views
Where to find a list of all the operable exploration / mining companies conducting operations in Canada [closed]
Looking to create a database firstly of all the exploration / mining companies conducting research in Canada. TSX and TSX-V have very nice spreadsheets with several filtering operations, which is very ...
- 11
4
votes
3
answers
5k
views
How to structure a stock market data database
I would like to download stock market data from the internet (for example by scraping…) and organize them in a database (I am using python and SQL) which updates daily or on request.
(The idea is to ...
- 119
1
vote
1
answer
67
views
should I persist the daily/weekly bar data
Suppose a small quant group (4 guys) and the tick data is in hand, shall they persist the historical daily/weekly or even smaller scale like hourly/minutes bars data, or just do the on-demand ...
- 11
2
votes
0
answers
175
views
Historical Credit Ratings for companies
I am looking forward to get my master thesis done. I have acces to Thomson Reuters Eikon. Unfortunately, they do not provide a good database with historical credit ratings - at least it is not ...
- 21
2
votes
1
answer
126
views
Any database with all of companies previous CEO's?
I'm doing some research on CEO's and company chairmen and I'm looking for a database that contains this data for all the public US companies. The data needs dates associated with the names as well. I'...
- 73
0
votes
1
answer
95
views
TS Database performance issue
I've just found out about this forum while searching for answers for an InfluxDB performance "issue". I'm using it to store financial tick data (7 fields per row), query it and process it into candles....
- 1
1
vote
1
answer
848
views
Orderbook db structure
I am currently saving a sub 1 sec snapshot of an orderbook to my SQL db. However I have quite the trouble on figuring out the architecture of this DB
What I'm currently doing is saving a table with ...
- 13
1
vote
1
answer
153
views
suggestions for improving monitoring of trading bots?
I'm simply looking for tips and ideas to make my system a little more professional.
The structure: The botting script is hosted on a VPS. There's a database hosted on yet another VPS. There's a ...
user41312
1
vote
0
answers
45
views
Analyst Calls Conference Transcript Database/API
Do you know any conference call transcript database/api ?
I already found one from Morningstar.
Thank you.
- 319
4
votes
1
answer
2k
views
how to treat NA values in Compustat and CRSP
In some years, some accounting values of some companies are missing in Compustat and CRSP. How do I treat those missing values ? Should I replace those with zeros or just simply delete years in which ...
- 117
1
vote
0
answers
290
views
How to handle database updates for splits/dividends?
I'm finding it an difficult task to maintain a database of stock prices. My main problem is how to efficiently handle splits and dividends. Is it better to handle this in a database with adjusted ...
- 23
1
vote
0
answers
233
views
Any databases with CEO letters and shareholder letters?
I am researching disclosures on CEO letters and Shareholder letters present in annual reports of listed companies in the London Stock Exchange.
Can someone recommend a database where the annual ...
- 111
0
votes
2
answers
160
views
What's the most acceptable name for the "put or call" attribute of an option contract?
Very mundane question. My team is setting up a security master db that includes listed option contracts. I'd like to store the "put or call" attribute in a well-named field, one that won't be ...
- 53
-1
votes
1
answer
1k
views
Where to find historical fundamental data of S&P constituents in Thomson Reuters database?
I need data such as Net Income, ROA, ROE, etc. for companies in S&P 500 Index. I would like to see the values also for other years, e.g., since 2010. However, when I log-in to Thomson Reuters ...
- 1
1
vote
1
answer
472
views
How to cite quandl data
I downloaded some stock quotes from Quandl databases 'Wiki EOD Stock Prices' and '
YFinance'. Now I want to report the result of calculations on this data in the publication. What is the correct way ...
2
votes
4
answers
5k
views
Close or Adjusted Prices when Backtesting
I've been doing this for some years now, but recently, since I started fiddling around with an old pairs trading strat of mine again, when updating the databases before running the tests, I was ...
1
vote
1
answer
910
views
Which data to use with the Fama French 5 factors model & q factor model
I'm working on my thesis in asset pricing, particularly on 'Fama French 5 factors model & q factor model'. May I know why I have to work with the FBRIT and DEADUK securities list for the UK market ...
5
votes
0
answers
213
views
What are some standard software stacks for financial data collection, storage and visualization?
I'm looking for the fastest way to get up-and-running collecting time-series data (mainly through scraping and through 3rd party APIs), storing it and visualizing it on a dashboard with graphs.
I'd ...
90
votes
10
answers
50k
views
Building Financial Data Time Series Database from scratch
My company is starting a new initiative aimed at building a financial database from scratch.
We would be using it in these ways:
Time series analysis of: a company's financial data (ex: IBM's total ...
- 1,003
0
votes
1
answer
3k
views
Where to source security ID data (ISIN, CUSIP)?
I am lookign for a list of ISINs or CUSIP for all equity securities traded on the NASDAQ and NYSE.
while not quantitative, this question is directly related to the data collection process of many ...
- 210
0
votes
3
answers
2k
views
Where can I find CMS swap trading prices?
I am writing a paper about CMS swap. To do so, I'd like to compare different theoretical pricing methods of these instruments to the "real prices" i.e. prices used in the marketplace.
But I don't ...
- 800
11
votes
3
answers
14k
views
Which Database (MySql or NoSQL) for a Stock market App
I'm re-creating an app for Stockmarket Screening & Realtime charting Display.
The database wireframe which i propose to design is as follows:
1. Company master - Where all the information of ...
- 111
2
votes
1
answer
153
views
Where to find Investment home bias data & Historical country weight in World MSCI
I need the following data and struggles to find it, maybe some of you can help me. Note: I'm a student and in our university ain't Bloomberg nor Reuters.
Investment home bias, e.g in 2012 US ...
- 257
8
votes
3
answers
2k
views
Creating Options Database
I am trying to create a database which will hold information for various stock options and will need to be updated daily. The idea is to use this database to keep track of changes in the open interest ...
- 81
8
votes
4
answers
12k
views
Which database to choose for storing and aggregating finance data?
I'm planing to store stock market data in realtime and aggregate ticks for draw volume based cluster graph. Something like this:
Every tick (or second) data will be grouped by period (1,5,10 minutes; ...
- 181
3
votes
2
answers
3k
views
Free high resolution financial data
As thebonnotgang(1) stopped updating their database, I was wondering if there are some other free sources of high-frequency data available.
I found a proper tick data api (ca. 25 day history) hosted ...
- 31
16
votes
2
answers
11k
views
What time series database can be used with Python and Pandas?
I'm looking for a time series database that can be easily used with Python and Pandas objects such as DataFrame, Panel...
But these objects will always contains time series.
Ideally I'm looking for ...
- 363
32
votes
4
answers
19k
views
Performance of Open Source Time Series Database for Financial Market Data
We would like to store financial tick data in a database (potentially billions of rows) and then create aggregated (open-high-low-close) bar data from it (e.g. 1min or 5min bars).
It was mentioned ...
- 697
3
votes
1
answer
821
views
Kenneth R. French data base on momentum and size: construction and how to use it concretely with momentum only
So, you can find all the data bases on this site. More explicitly, I would like to take the "Developed Market Factors and Returns" part. Even more explicitly, let us take the "25 Portfolios Formed on ...
- 299
6
votes
1
answer
1k
views
SEC 13F Security List has incorrect CUSIP numbers?
I'm building database of 13F sec.gov forms with 13F security lists (http://www.sec.gov/divisions/investment/13flists.htm). Along with integrity checks.
I implemented CUSIP digit check algorithm to ...
- 198
1
vote
1
answer
1k
views
Where can I find US public company bankruptcy data
I am doing a thesis about firm survival in time of crisis (2008-2009) and I would like to know where can I find publicly available database about company bankruptcy.
Since I have thousands of ...
- 11
2
votes
1
answer
279
views
Create 10-K Filing Database
I am finance grad student, and working to create a financial statement database. My objective is to download an income statement, balance sheet, and statement of cash flows for every publicly traded ...
- 123
3
votes
3
answers
4k
views
float64 to store price data: is precision sufficient?
I am looking to store equity price data in a hdf5 table. The use will be purely as a historical archive, not as day-to-day data source.
Options
One option would be to store base10 significand and ...
- 65
0
votes
3
answers
3k
views
Databases for storing and querying high frequency tick-level data?
My firm is looking for an out-of-the-box database system to store and query high-frequency tick data. What are the best options? It seems that kdb+ is the market leader in this field.
- 109
3
votes
2
answers
650
views
Places to make quant code/tools publicly avaliable
Over the years I have developed several tools - including pricing, optimization and calibration tools - most in VBA, C# and C++
I would like to make them publicly avaliable. Aside from putting up my ...
- 3,357
1
vote
2
answers
11k
views
S&P's Sovereign Ratings: Clarification on Definitions and Symbols
Similar to an earlier question, I am now looking at S&P's sovereign ratings. Here, as in the case of Moody's, a few things are unclear to me in terms of the definitions used by Standard & Poor'...
- 237
2
votes
1
answer
1k
views
Moody's Sovereign Ratings: Clarification on Definitions and Symbols
I'm working with sovereign ratings at present. With regard to Moody's there are a few things unclear to me in their definitions. Both questions refer to the sovereign rating history in Bloomberg CSDR (...
- 237