Skip to main content
108 votes
Accepted

Building Financial Data Time Series Database from scratch

I am going to recommend something that I have no doubt will get people completely up in arms and probably get people to attack me. It happened in the past and I lost many points on StackOverflow as ...
drobertson's user avatar
  • 1,882
35 votes

Building Financial Data Time Series Database from scratch

All of the answers above (unfortunately highly upvoted at this point) are missing the point. You shouldn't pick a DBMS or storage solution by general performance benchmarks, you should pick it by use ...
madilyn's user avatar
  • 5,240
16 votes

Building Financial Data Time Series Database from scratch

Interesting debate and Not to wake sleeping dogs, the world has moved quite a bit in the 1.5 years, and the data space has exploded. I would like to recommend some ...
chjortlund's user avatar
15 votes

Building Financial Data Time Series Database from scratch

The standard answer is going to be that for time series, you want a column store database. These are optimized for range queries (ie: give me everything between two timestamps) because crucially, they ...
Thomas Browne's user avatar
13 votes
Accepted

How to find the formula for the half-life of an AR(1) process (using the Ornstein–Uhlenbeck process)

Convenient rewriting Let $$X_t = c + \phi_1 X_{t-1} + \epsilon_t, \quad \vert \phi_1 \vert < 1 \tag{1} $$ denote a weakly stationary AR(1) process. Weak stationarity notably implies that $$\Bbb{E}[...
Quantuple's user avatar
  • 14.7k
13 votes

Choosing the right statistical test for Mutual Fund Performance Evaluation

Define excess return $r^x_{it} = r_{it} - r^f_{t}$ as the return $i$ minus the risk free rate, and $f_{jt}$ similarly denotes the excess return of factor $j$ at time $t$. Let's say we have some factor ...
Matthew Gunn's user avatar
  • 6,974
13 votes

Why quants think that the risk-neutral measure should not be used for financial forecasting?

There is a deeper issue. Frequentist distributions are not probability distributions because they are designed to be minimax distributions rather than actual distributions. This ignores all of the ...
Dave Harris's user avatar
  • 4,299
11 votes
Accepted

Volume or Dollar bars vs. volatility normalized and demeaned financial time series

The dollar bars certainly allow for a partial recovery of normality through a price sampling process subordinated to a volume, tick, dollar clock. It is well known that returns are assumed to be ...
Jacques Joubert's user avatar
10 votes
Accepted

Realized Variance (realized volatility)

The TLDR; to your question: How can one use realized volatility as a volatility model to do out-of-sample prediction? You extend known models to incorporate additional information procured from high-...
Pleb's user avatar
  • 4,486
10 votes
Accepted

Daily realized volatility and true daily volatility

To keep it brief: the realized variance estimator, $RV_t$, is only a consistent estimator of Quadratic Variation (QV) under absence of microstructure noise. Following the paper of Barndorff‐Nielsen, O....
Pleb's user avatar
  • 4,486
9 votes

Why quants think that the risk-neutral measure should not be used for financial forecasting?

In their book "Counterparty Credit Risk, Collateral and Funding" D. Brigo, M. Morini and A. Pallavicini start with a dialogue between a Physics PhD graduate and an experienced practitioner of ...
Nicolas Gutierrez's user avatar
9 votes

What is the purpose of short rate models?

Short rate models were first used in the 1970s and 1980s to try to fit and explain the term structure of interest rates - they went beyond simple parametric shapes (polynomials and exponential forms). ...
Dom's user avatar
  • 2,167
9 votes
Accepted

Is there a HAR that deals with the leverage effect?

There exists a modification of the HAR model that accounts for leverage effect (á la GJR-GARCH) in a high-frequency setting. The semi-variance HAR model, termed the SHAR model of Patton and Sheppard (...
Pleb's user avatar
  • 4,486
8 votes

Why quants think that the risk-neutral measure should not be used for financial forecasting?

Perhaps a case of views based upon theoretical possibilities rather than empirical realities? In theory, $P$ and $Q$ can be extremely different $P$ is the real world, actual probability measure. $Q$ ...
Matthew Gunn's user avatar
  • 6,974
7 votes
Accepted

Does the unconditional variance implied by a GARCH equal the sample variance?

In this context, unconditional variance refers to the stationary variance level predicted by your GARCH model. This quantity need not coincide with the sample variance of the data on which the latter ...
Quantuple's user avatar
  • 14.7k
7 votes
Accepted

Why is the GARCH intercept supposed to be strictly positive?

Consider the GARCH(1,1) process \begin{align} r_{t+1} &= \sigma_{t+1} z_{t+1} \\ \sigma^2_{t+1} &= \omega+\alpha r^2_t +\beta \sigma^2_{t} \end{align} for the returns $r_t$, with ${z_t} \sim ...
Quantuple's user avatar
  • 14.7k
7 votes
Accepted

How to deal with missing value in a time series stock market data?

Some approaches Use only common points - Exclude all holidays in any index. Reduced sample size Loss of information No 'made up' data (consistency) Fill forward - use previous day as you ...
user25064's user avatar
  • 1,077
7 votes
Accepted

How to perform cross-sectional asset pricing regression?

I prefer thinking in terms of well measured vs. poorly measured rather than significant vs. insignificant: arbitrary p-value cutoffs and ignoring sensible priors can both be problematic. On the ...
Matthew Gunn's user avatar
  • 6,974
7 votes
Accepted

Is there a way to tell if a time series price data is reversed?

This is a great question! The simple, boring answer is clearly "no" - given a time series of stock prices, there is no way to tell for certain whether the price series has been reversed or not. For ...
Chris Taylor's user avatar
  • 5,931
7 votes
Accepted

Converting time bars to tick bars or volume bars in python

As mentioned in my comment, tick data is the individual quotes and trades; Yahoo only has daily data. As an analogy, you can always make a high-definition photo more blurry and pixelated, but you can'...
chrisaycock's user avatar
  • 9,837
7 votes

What color financial time series are there?

Bonus question: Does anyone know how to play/hear a (financial) time series recorded as a pandas series, dataframe, python list, numpy array, csv/txt file,... ? This is kind of fun and has practical ...
amdopt's user avatar
  • 4,348
6 votes

Is R being replaced by Python at quant desks?

Also in the high frequency / medium frequency field here. I received a "mixed" consensus regarding the use of R and its prevalence in the field (specifically HFT). Speaking with someone who works in ...
Theodore's user avatar
  • 1,172
6 votes

How to fit ARMA+GARCH Model In R?

This should walk you through what you are looking for: https://www.quantstart.com/articles/Generalised-Autoregressive-Conditional-Heteroskedasticity-GARCH-p-q-Models-for-Time-Series-Analysis https://...
cJc's user avatar
  • 207
6 votes

Imposing Restrictions on Cointegrating Vectors, R example

I know this was asked almost two years ago, but I thought I'd answer the question. It appears that the H that you want to estimate is identical to the values you received from the Johansen test, ...
Cameron Pfiffer's user avatar
6 votes
Accepted

Reference request: Quantitative approaches to market abuse detection

I can't help as much with public literature, but I did see a talk with a member of the FINRA data science team responsible for exactly this (event link below - perhaps you can track down the speaker). ...
mperlow's user avatar
  • 456
6 votes

Why quants think that the risk-neutral measure should not be used for financial forecasting?

The risk neutral density is a mathematical trick to allow pricing of options. As it has little bearing on reality, it makes little sense to simulate from it for the purposes of forecasting real ...
user9403's user avatar
  • 1,429
6 votes
Accepted

Asset pricing model factor need to be excess return?

AP factors do not need to be excess returns. In case they are, corresponding prices of risk are conveniently equal to average factor values, since "factors price themselves": $$E[R_i] = \beta_{i} \...
Igor Pozdeev's user avatar
5 votes

Is a linear combination of GARCH processes also a GARCH process?

No, a sum of two GARCH processes is generally not a GARCH process. (I am not even sure whether there exists a nontrivial special case where the opposite holds.) By GARCH I mean the classic ...
Richard Hardy's user avatar
5 votes

What machine learning method is more suitable for prediction of financial time series?

From what I have read, there are 3 popular algorithms for financial time series. Random Forests and SVMs, then followed by Neural Network Architectures. There are a couple of good papers, to name a ...
Jacques Joubert's user avatar
5 votes
Accepted

How to obtain Standardized Residuals from a Time-Series?

Usually for MLE estimation as you said we compute the residuals starting from index number of lag+1 (p+1 for AR model) in this case we obtain Conditional MLE ...
Malick's user avatar
  • 2,582

Only top scored, non community-wiki answers of a minimum length are eligible