106
votes
Accepted
Building Financial Data Time Series Database from scratch
I am going to recommend something that I have no doubt will get people completely up in arms and probably get people to attack me. It happened in the past and I lost many points on StackOverflow as ...
- 1,862
35
votes
Building Financial Data Time Series Database from scratch
All of the answers above (unfortunately highly upvoted at this point) are missing the point. You shouldn't pick a DBMS or storage solution by general performance benchmarks, you should pick it by use ...
- 5,231
32
votes
Is R being replaced by Python at quant desks?
Instead of wild guesses about R's/python's future in the community, here some facts:
The following query on StackExchange Data Explorer counts the number of questions that have ...
- 481
16
votes
Building Financial Data Time Series Database from scratch
Interesting debate and Not to wake sleeping dogs, the world has moved quite a bit in the 1.5 years, and the data space has exploded.
I would like to recommend some ...
- 680
15
votes
Performance of Open Source Time Series Database for Financial Market Data
You could try Arctic. Other open source column-oriented databases that you may not have considered include LucidDB and C-Store.
- 5,231
15
votes
Building Financial Data Time Series Database from scratch
The standard answer is going to be that for time series, you want a column store database. These are optimized for range queries (ie: give me everything between two timestamps) because crucially, they ...
- 1,150
14
votes
How to calculate the conditional variance of a time series?
Let’s take a simple example to answer a broad but interesting question:
Imagine that we have a daily return serie denoted $r_{t}$ ( which is assumed to be stationary) and let's take a little time to ...
- 2,542
13
votes
Choosing the right statistical test for Mutual Fund Performance Evaluation
Define excess return $r^x_{it} = r_{it} - r^f_{t}$ as the return $i$ minus the risk free rate, and $f_{jt}$ similarly denotes the excess return of factor $j$ at time $t$. Let's say we have some factor ...
- 6,864
13
votes
Why quants think that the risk-neutral measure should not be used for financial forecasting?
There is a deeper issue. Frequentist distributions are not probability distributions because they are designed to be minimax distributions rather than actual distributions. This ignores all of the ...
- 4,289
12
votes
Accepted
Relationships between white noise and random walk
I will assume a white noise is a process $(\varepsilon_t)$ with zero mean, no autocorrelation and constant variance $\sigma^2 > 0$ while a random walk is a process $(x_t)$ defined by
$$
x_{t+1} =...
- 3,926
11
votes
Accepted
How to find the formula for the half-life of an AR(1) process (using the Ornstein–Uhlenbeck process)
Convenient rewriting
Let $$X_t = c + \phi_1 X_{t-1} + \epsilon_t, \quad \vert \phi_1 \vert < 1 \tag{1} $$
denote a weakly stationary AR(1) process. Weak stationarity notably implies that $$\Bbb{E}[...
- 14.5k
11
votes
Accepted
Volume or Dollar bars vs. volatility normalized and demeaned financial time series
The dollar bars certainly allow for a partial recovery of normality through a price sampling process subordinated to a volume, tick, dollar clock.
It is well known that returns are assumed to be ...
- 924
10
votes
Accepted
Realized Variance (realized volatility)
The TLDR; to your question:
How can one use realized volatility as a volatility model to do out-of-sample prediction? You extend known models to incorporate additional information procured from high-...
- 4,156
9
votes
What time series database can be used with Python and Pandas?
OpenTSDB is good for large-scale time series storage. metrilyx/opentsdb-pandas and wiktorski/opentsdb_pandas seems to provide the interface with pandas. OpenTSDB and HBase rough performance test | ...
- 246
9
votes
Why quants think that the risk-neutral measure should not be used for financial forecasting?
In their book "Counterparty Credit Risk, Collateral and Funding" D. Brigo, M. Morini and A. Pallavicini start with a dialogue between a Physics PhD graduate and an experienced practitioner of ...
9
votes
What is the purpose of short rate models?
Short rate models were first used in the 1970s and 1980s to try to fit and explain the term structure of interest rates - they went beyond simple parametric shapes (polynomials and exponential forms). ...
- 2,137
9
votes
Accepted
Is there a HAR that deals with the leverage effect?
There exists a modification of the HAR model that accounts for leverage effect (á la GJR-GARCH) in a high-frequency setting.
The semi-variance HAR model, termed the SHAR model of Patton and Sheppard (...
- 4,156
9
votes
Accepted
Daily realized volatility and true daily volatility
To keep it brief: the realized variance estimator, $RV_t$, is only a consistent estimator of Quadratic Variation (QV) under absence of microstructure noise.
Following the paper of Barndorff‐Nielsen, O....
- 4,156
8
votes
Why quants think that the risk-neutral measure should not be used for financial forecasting?
Perhaps a case of views based upon theoretical possibilities rather than empirical realities?
In theory, $P$ and $Q$ can be extremely different
$P$ is the real world, actual probability measure.
$Q$ ...
- 6,864
8
votes
What color financial time series are there?
Bonus question: Does anyone know how to play/hear a (financial) time series recorded as a pandas series, dataframe, python list, numpy array, csv/txt file,... ?
This is kind of fun and has practical ...
- 4,690
7
votes
Accepted
Shannon's entropy for financial times-series (return)
What you need is more mutual information rather than Shannon entropy. It is dedicated to capture the influence of one variable on another (you can think about it as a non linear version of Pearson ...
- 11.4k
7
votes
Accepted
Does the unconditional variance implied by a GARCH equal the sample variance?
In this context, unconditional variance refers to the stationary variance level predicted by your GARCH model. This quantity need not coincide with the sample variance of the data on which the latter ...
- 14.5k
7
votes
Accepted
Why is the GARCH intercept supposed to be strictly positive?
Consider the GARCH(1,1) process
\begin{align}
r_{t+1} &= \sigma_{t+1} z_{t+1} \\
\sigma^2_{t+1} &= \omega+\alpha r^2_t +\beta \sigma^2_{t}
\end{align}
for the returns $r_t$, with ${z_t} \sim ...
- 14.5k
7
votes
Accepted
How to deal with missing value in a time series stock market data?
Some approaches
Use only common points - Exclude all holidays in any index.
Reduced sample size
Loss of information
No 'made up' data (consistency)
Fill forward - use previous day as you ...
- 1,047
7
votes
Accepted
How to perform cross-sectional asset pricing regression?
I prefer thinking in terms of well measured vs. poorly measured rather than significant vs. insignificant: arbitrary p-value cutoffs and ignoring sensible priors can both be problematic. On the ...
- 6,864
7
votes
Accepted
Is there a way to tell if a time series price data is reversed?
This is a great question!
The simple, boring answer is clearly "no" - given a time series of stock prices, there is no way to tell for certain whether the price series has been reversed or not. For ...
- 5,798
6
votes
How to fit ARMA+GARCH Model In R?
This should walk you through what you are looking for:
https://www.quantstart.com/articles/Generalised-Autoregressive-Conditional-Heteroskedasticity-GARCH-p-q-Models-for-Time-Series-Analysis
https://...
- 207
6
votes
Is R being replaced by Python at quant desks?
Also in the high frequency / medium frequency field here.
I received a "mixed" consensus regarding the use of R and its prevalence in the field (specifically HFT). Speaking with someone who works in ...
- 1,172
6
votes
Does Kalman filter always improve over linear regression?
There is no a "yes/no answer" to that question. Generally Kalman Filter tends to be better than linear regression, but everything depends on
the data which you have,
how you calibrate your model.
...
6
votes
Imposing Restrictions on Cointegrating Vectors, R example
I know this was asked almost two years ago, but I thought I'd answer the question.
It appears that the H that you want to estimate is identical to the values you received from the Johansen test, ...
- 116
Only top scored, non community-wiki answers of a minimum length are eligible