Questions tagged [database]

A structured collection of data, with meta-data describing is content and meaning. Also for database management systems (DBMS) which are software products for managing and manipulating that data.

Filter by
Sorted by
Tagged with
0
votes
3answers
2k views

Databases for storing and querying high frequency tick-level data?

My firm is looking for an out-of-the-box database system to store and query high-frequency tick data. What are the best options? It seems that kdb+ is the market leader in this field.
0
votes
0answers
71 views

Why do researchers exclude firms with total asset less than £1 million?

Chaney, Jeter, and Shivakumar (2004, page 58) exclude firms with less than 1 million pound in total assets Further, we exclude firms with less than £1 million in total assets I am wondering why they ...
1
vote
0answers
58 views

Are there any best practices for designing high frequency trading systems?

I spent some time trying to design some parts of the system, going over the information I found. At the top-level, the system looks like this A "real-time" module that receives market data, ...
0
votes
2answers
61 views

Where can I get fundamental data on US public stocks in original form?

I'm looking for fundamental data on US public companies. If anyone could recommend paid or free sources of the fundamentals in original, non-standardized form that would be great. I tried scraping the ...
1
vote
0answers
65 views

Which Times Series Database framework for Python is best for portfolio optimization project?

I am starting to build a portfolio optimization algorithm in Python and want to structure a database to manipulate financial data. Although I have Python experience, I have never used SQL or such ...
49
votes
18answers
50k views

Is there any thing out there as a substitute for KDB?

thanks a lot for your discussions on the original post. following your suggestions, let me re-phrase a bit : kdb is known for its efficiency, and such efficiency comes at a terrible price. However, ...
0
votes
1answer
37 views

should I persist the daily/weekly bar data

Suppose a small quant group (4 guys) and the tick data is in hand, shall they persist the historical daily/weekly or even smaller scale like hourly/minutes bars data, or just do the on-demand ...
1
vote
0answers
26 views

From SEC Comprehensive Data Set to Clean Balance Sheets/Income Statements

I am looking for advice on how to smartly get from unstructured financial data to a clean summary of balance sheets. I think of this not as a coding problem but of a question how to approach the task -...
10
votes
1answer
800 views

What's the most efficient way to store options and time series data for backtesting?

I would like to know what database would you guys use for storing around 500GB-1TB of options and time series tick data. The idea is to use it for backtesting so it would have to be as efficient as ...
10
votes
3answers
11k views

Which Database (MySql or NoSQL) for a Stock market App

I'm re-creating an app for Stockmarket Screening & Realtime charting Display. The database wireframe which i propose to design is as follows: 1. Company master - Where all the information of ...
0
votes
0answers
40 views

Question about the coverage of Datastream compared to WorldScope regarding US data

Thomson Reuters Datastream is claimed as a very comprehensive data source. I did some comparisons and also conducted research myself and cast doubt on that, especially for US data. I used the paper of ...
1
vote
0answers
18 views

Where to find a list of all the operable exploration / mining companies conducting operations in Canada [closed]

Looking to create a database firstly of all the exploration / mining companies conducting research in Canada. TSX and TSX-V have very nice spreadsheets with several filtering operations, which is very ...
0
votes
0answers
24 views

Derivatives notional amount in BankFocus

I'm searching for the notional amount of derivatives in banks' balance sheets. I'm using "BankFocus" (ex Bankscope) to download data. In balance sheet i find just the fair value of ...
2
votes
3answers
4k views

Close or Adjusted Prices when Backtesting

I've been doing this for some years now, but recently, since I started fiddling around with an old pairs trading strat of mine again, when updating the databases before running the tests, I was ...
1
vote
3answers
583 views

How to structure a stock market data database

I would like to download stock market data from the internet (for example by scraping…) and organize them in a database (I am using python and SQL) which updates daily or on request. (The idea is to ...
78
votes
9answers
43k views

Building Financial Data Time Series Database from scratch

My company is starting a new initiative aimed at building a financial database from scratch. We would be using it in these ways: Time series analysis of: a company's financial data (ex: IBM's total ...
30
votes
4answers
18k views

Performance of Open Source Time Series Database for Financial Market Data

We would like to store financial tick data in a database (potentially billions of rows) and then create aggregated (open-high-low-close) bar data from it (e.g. 1min or 5min bars). It was mentioned ...
1
vote
0answers
46 views

Historical Credit Ratings for companies

I am looking forward to get my master thesis done. I have acces to Thomson Reuters Eikon. Unfortunately, they do not provide a good database with historical credit ratings - at least it is not ...
1
vote
1answer
49 views

Any database with all of companies previous CEO's?

I'm doing some research on CEO's and company chairmen and I'm looking for a database that contains this data for all the public US companies. The data needs dates associated with the names as well. I'...
36
votes
7answers
24k views

Usage of NoSQL storage in Finance

I am wondering if anyone has used NoSQL (mongodb, cassandra, etc.) to store and analyze data. I tried searching the web but was not able to see if the financial firms had gotten in to using nosql ...
0
votes
1answer
66 views

TS Database performance issue

I've just found out about this forum while searching for answers for an InfluxDB performance "issue". I'm using it to store financial tick data (7 fields per row), query it and process it into candles....
1
vote
1answer
329 views

Orderbook db structure

I am currently saving a sub 1 sec snapshot of an orderbook to my SQL db. However I have quite the trouble on figuring out the architecture of this DB What I'm currently doing is saving a table with ...
7
votes
3answers
1k views

Creating Options Database

I am trying to create a database which will hold information for various stock options and will need to be updated daily. The idea is to use this database to keep track of changes in the open interest ...
1
vote
1answer
126 views

suggestions for improving monitoring of trading bots?

I'm simply looking for tips and ideas to make my system a little more professional. The structure: The botting script is hosted on a VPS. There's a database hosted on yet another VPS. There's a ...
0
votes
3answers
2k views

Where can I find CMS swap trading prices?

I am writing a paper about CMS swap. To do so, I'd like to compare different theoretical pricing methods of these instruments to the "real prices" i.e. prices used in the marketplace. But I don't ...
-2
votes
3answers
3k views

Where to get access to an inexpensive or free hedge fund/CTA DB?

Basically, the question is in the title. For some similar data: EDHEC provides some data and one can also get the Mutual Funds data from the Yahoo Finance:is there anything else out there? Thanks,
1
vote
0answers
37 views

Analyst Calls Conference Transcript Database/API

Do you know any conference call transcript database/api ? I already found one from Morningstar. Thank you.
4
votes
1answer
1k views

how to treat NA values in Compustat and CRSP

In some years, some accounting values of some companies are missing in Compustat and CRSP. How do I treat those missing values ? Should I replace those with zeros or just simply delete years in which ...
1
vote
0answers
211 views

How to handle database updates for splits/dividends?

I'm finding it an difficult task to maintain a database of stock prices. My main problem is how to efficiently handle splits and dividends. Is it better to handle this in a database with adjusted ...
1
vote
0answers
171 views

Any databases with CEO letters and shareholder letters?

I am researching disclosures on CEO letters and Shareholder letters present in annual reports of listed companies in the London Stock Exchange. Can someone recommend a database where the annual ...
-1
votes
1answer
1k views

Where to find historical fundamental data of S&P constituents in Thomson Reuters database?

I need data such as Net Income, ROA, ROE, etc. for companies in S&P 500 Index. I would like to see the values also for other years, e.g., since 2010. However, when I log-in to Thomson Reuters ...
-4
votes
1answer
190 views

API to perform queries on stocks and financial derivatives [closed]

Is there a native and free API and DB to request information such as: All the companies which their P/E ratio is greater than 50 The closing prices of stock <...
0
votes
2answers
154 views

What's the most acceptable name for the "put or call" attribute of an option contract?

Very mundane question. My team is setting up a security master db that includes listed option contracts. I'd like to store the "put or call" attribute in a well-named field, one that won't be ...
1
vote
1answer
772 views

Which data to use with the Fama French 5 factors model & q factor model

I'm working on my thesis in asset pricing, particularly on 'Fama French 5 factors model & q factor model'. May I know why I have to work with the FBRIT and DEADUK securities list for the UK market ...
0
votes
2answers
9k views

S&P's Sovereign Ratings: Clarification on Definitions and Symbols

Similar to an earlier question, I am now looking at S&P's sovereign ratings. Here, as in the case of Moody's, a few things are unclear to me in terms of the definitions used by Standard & Poor'...
1
vote
1answer
452 views

How to cite quandl data

I downloaded some stock quotes from Quandl databases 'Wiki EOD Stock Prices' and ' YFinance'. Now I want to report the result of calculations on this data in the publication. What is the correct way ...
2
votes
3answers
508 views

Options Data Sources

I am using Option Metrics to study a couple of things related to options. However, Option Metrics is quite limited in terms of scope (mainly it's US equities). I was wondering two things: 1) Are ...
13
votes
2answers
6k views

Why use a column database for tick/bar data?

I often hear that column-oriented databases are the best choice method for storing time series data in finance applications. Especially by people selling expensive column-oriented databases. Yet, at ...
5
votes
0answers
198 views

What are some standard software stacks for financial data collection, storage and visualization?

I'm looking for the fastest way to get up-and-running collecting time-series data (mainly through scraping and through 3rd party APIs), storing it and visualizing it on a dashboard with graphs. I'd ...
80
votes
9answers
37k views

Efficiently storing real-time intraday data in an application agnostic way

What would be the best approach to handle real-time intraday data storage? For personal research I've always imported from flat files only into memory (historical EOD), so I don't have much ...
3
votes
3answers
3k views

float64 to store price data: is precision sufficient?

I am looking to store equity price data in a hdf5 table. The use will be purely as a historical archive, not as day-to-day data source. Options One option would be to store base10 significand and ...
0
votes
1answer
2k views

Where to source security ID data (ISIN, CUSIP)?

I am lookign for a list of ISINs or CUSIP for all equity securities traded on the NASDAQ and NYSE. while not quantitative, this question is directly related to the data collection process of many ...
8
votes
4answers
11k views

Which database to choose for storing and aggregating finance data?

I'm planing to store stock market data in realtime and aggregate ticks for draw volume based cluster graph. Something like this: Every tick (or second) data will be grouped by period (1,5,10 minutes; ...
3
votes
2answers
3k views

Free high resolution financial data

As thebonnotgang(1) stopped updating their database, I was wondering if there are some other free sources of high-frequency data available. I found a proper tick data api (ca. 25 day history) hosted ...
15
votes
5answers
9k views

Best way to store hourly/daily options data for research purposes

There are quite a few discussions here about storage, but I can't find quite what I'm looking for. I'm in need to design a database to store (mostly) option data (strikes, premiums bid / ask, etc.). ...
2
votes
1answer
141 views

Where to find Investment home bias data & Historical country weight in World MSCI

I need the following data and struggles to find it, maybe some of you can help me. Note: I'm a student and in our university ain't Bloomberg nor Reuters. Investment home bias, e.g in 2012 US ...
3
votes
2answers
630 views

Places to make quant code/tools publicly avaliable

Over the years I have developed several tools - including pricing, optimization and calibration tools - most in VBA, C# and C++ I would like to make them publicly avaliable. Aside from putting up my ...
13
votes
2answers
10k views

What time series database can be used with Python and Pandas?

I'm looking for a time series database that can be easily used with Python and Pandas objects such as DataFrame, Panel... But these objects will always contains time series. Ideally I'm looking for ...
3
votes
1answer
676 views

Kenneth R. French data base on momentum and size: construction and how to use it concretely with momentum only

So, you can find all the data bases on this site. More explicitly, I would like to take the "Developed Market Factors and Returns" part. Even more explicitly, let us take the "25 Portfolios Formed on ...
1
vote
1answer
1k views

Where can I find US public company bankruptcy data

I am doing a thesis about firm survival in time of crisis (2008-2009) and I would like to know where can I find publicly available database about company bankruptcy. Since I have thousands of ...