104 votes

Building Financial Data Time Series Database from scratch

I am going to recommend something that I have no doubt will get people completely up in arms and probably get people to attack me. It happened in the past and I lost many points on StackOverflow as ...
user avatar
  • 1,807
49 votes
Accepted

Is R being replaced by Python at quant desks?

My deal is HFT so what I care about is read/load data from file or DB quickly in memory perform very efficient data-munging operations (group,transform) visualize easily the data I think is is ...
user avatar
  • 1,248
34 votes

Building Financial Data Time Series Database from scratch

All of the answers above (unfortunately highly upvoted at this point) are missing the point. You shouldn't pick a DBMS or storage solution by general performance benchmarks, you should pick it by use ...
user avatar
  • 5,062
32 votes

Is R being replaced by Python at quant desks?

Instead of wild guesses about R's/python's future in the community, here some facts: The following query on StackExchange Data Explorer counts the number of questions that have ...
user avatar
  • 491
26 votes

Is R being replaced by Python at quant desks?

This is interesting because I see another trend: Matlab is being replaced by R, but I guess this is another story. I use R for my academic (I am also teaching this stuff) as well as my consulting ...
user avatar
  • 27k
23 votes

Is R being replaced by Python at quant desks?

I've used both R and Python with Pandas in a professional quantitative financial work to do both large and small scale projects. I would strongly recommend Python with Pandas over R for most new ...
user avatar
  • 1,551
16 votes
Accepted

Why do we usually model returns and not prices?

Basically, prices usually have a unit root, while returns can be assumed to be stationary. This is also called order of integration, a unit root means integrated of order 1, I(1), while stationary is ...
user avatar
  • 5,311
15 votes

Performance of Open Source Time Series Database for Financial Market Data

You could try Arctic. Other open source column-oriented databases that you may not have considered include LucidDB and C-Store.
user avatar
  • 5,062
15 votes

Building Financial Data Time Series Database from scratch

The standard answer is going to be that for time series, you want a column store database. These are optimized for range queries (ie: give me everything between two timestamps) because crucially, they ...
user avatar
15 votes

Building Financial Data Time Series Database from scratch

Interesting debate and Not to wake sleeping dogs, the world has moved quite a bit in the 1.5 years, and the data space has exploded. I would like to recommend some ...
user avatar
13 votes

How to calculate the conditional variance of a time series?

Let’s take a simple example to answer a broad but interesting question: Imagine that we have a daily return serie denoted $r_{t}$ ( which is assumed to be stationary) and let's take a little time to ...
user avatar
  • 2,514
13 votes

Is R being replaced by Python at quant desks?

For data analysis, particularly for large data analysis project, pretty much most of the top quant hedge funds and a lot of the banks are using Python (over R) for a couple of reasons but many still ...
user avatar
  • 782
13 votes

Why quants think that the risk-neutral measure should not be used for financial forecasting?

There is a deeper issue. Frequentist distributions are not probability distributions because they are designed to be minimax distributions rather than actual distributions. This ignores all of the ...
user avatar
  • 4,105
12 votes
Accepted

Relationships between white noise and random walk

I will assume a white noise is a process $(\varepsilon_t)$ with zero mean, no autocorrelation and constant variance $\sigma^2 > 0$ while a random walk is a process $(x_t)$ defined by $$ x_{t+1} =...
user avatar
  • 3,836
12 votes

Choosing the right statistical test for Mutual Fund Performance Evaluation

Define excess return $r^x_{it} = r_{it} - r^f_{t}$ as the return $i$ minus the risk free rate, and $f_{jt}$ similarly denotes the excess return of factor $j$ at time $t$. Let's say we have some factor ...
user avatar
  • 6,364
11 votes
Accepted

Volume or Dollar bars vs. volatility normalized and demeaned financial time series

The dollar bars certainly allow for a partial recovery of normality through a price sampling process subordinated to a volume, tick, dollar clock. It is well known that returns are assumed to be ...
user avatar
10 votes

Why non-stationary data cannot be analyzed?

There is a lot of ways to understand why stationarity allows to apply usual time series analysis. Here is one more. Very often, the theoretical justification of what you do in time series need to be ...
user avatar
  • 10.6k
10 votes
Accepted

How to find the formula for the half-life of an AR(1) process (using the Ornstein–Uhlenbeck process)

Convenient rewriting Let $$X_t = c + \phi_1 X_{t-1} + \epsilon_t, \quad \vert \phi_1 \vert < 1 \tag{1} $$ denote a weakly stationary AR(1) process. Weak stationarity notably implies that $$\Bbb{E}[...
user avatar
  • 14k
9 votes

Book recommendation for time series analysis

I would suggest Time Series Analysis by James Douglas Hamilton
9 votes

What time series database can be used with Python and Pandas?

OpenTSDB is good for large-scale time series storage. metrilyx/opentsdb-pandas and wiktorski/opentsdb_pandas seems to provide the interface with pandas. OpenTSDB and HBase rough performance test | ...
user avatar
  • 246
9 votes

Why quants think that the risk-neutral measure should not be used for financial forecasting?

In their book "Counterparty Credit Risk, Collateral and Funding" D. Brigo, M. Morini and A. Pallavicini start with a dialogue between a Physics PhD graduate and an experienced practitioner of ...
user avatar
9 votes
Accepted

Is there a HAR that deals with the leverage effect?

There exists a modification of the HAR model that accounts for leverage effect (á la GJR-GARCH) in a high-frequency setting. The semi-variance HAR model, termed the SHAR model of Patton and Sheppard (...
user avatar
  • 3,753
9 votes
Accepted

Realized Variance (realized volatility)

The TLDR; to your question: How can one use realized volatility as a volatility model to do out-of-sample prediction? You extend known models to incorporate additional information procured from high-...
user avatar
  • 3,753
8 votes

Why do we usually model returns and not prices?

Perhaps overly simplistic and repeating the pt above, but when doing statistics, ideally we want to compare like with like. Returns can be comparable with each other. Prices on the other hand always ...
user avatar
8 votes
Accepted

Speed of mean reversion of an interest rate model

Mean reversion speed $\kappa$ is better interpreted with the concept of half-life, which can be calculated from $\text{HL} = \ln(2) / \kappa$. For example, if the mean reversion coefficient is $\kappa ...
user avatar
  • 10.9k
8 votes

Why quants think that the risk-neutral measure should not be used for financial forecasting?

Perhaps a case of views based upon theoretical possibilities rather than empirical realities? In theory, $P$ and $Q$ can be extremely different $P$ is the real world, actual probability measure. $Q$ ...
user avatar
  • 6,364
8 votes

What is the purpose of short rate models?

Short rate models were first used in the 1970s and 1980s to try to fit and explain the term structure of interest rates - they went beyond simple parametric shapes (polynomials and exponential forms). ...
user avatar
  • 2,069
7 votes

Which library shall I use for time series analysis in Java?

I work with time series intensively, and I am experienced in Java and scripting languages such as MATLAB and R. I strongly suggest that you should cook up your own implementations in Java, and stop ...
user avatar
  • 799
7 votes
Accepted

Shannon's entropy for financial times-series (return)

What you need is more mutual information rather than Shannon entropy. It is dedicated to capture the influence of one variable on another (you can think about it as a non linear version of Pearson ...
user avatar
  • 10.6k
7 votes
Accepted

Does the unconditional variance implied by a GARCH equal the sample variance?

In this context, unconditional variance refers to the stationary variance level predicted by your GARCH model. This quantity need not coincide with the sample variance of the data on which the latter ...
user avatar
  • 14k

Only top scored, non community-wiki answers of a minimum length are eligible