Regression model when samples are small and not correlated

Question

I received this question during an onsite interview for a quant job and I'm still scratching my head on how to solve this problem. Any help would be appreciated.

---

Mr Quant thinks that there is a linear relationship between past and future intraday returns. So he would like to test this idea. For convenience, he decided to parameterize return in his data set using a regular time grid dt where $d=0, …, D-1$ labels date and $t=0, …, T-1$ intraday time period. For example, if we split day into 10 minute intervals then $T = 1440 / 10$. His model written on this time grid has the following form:

$y_{d,t}$ $=$ $\beta_t$ * $x_{d,t}$ + $\epsilon_{d,t}$

where $y_{d,t}$ is a return over the time interval $(t,t+1)$ and $x_{d,t}$ is a return over the previous time interval, $(t–1,t)$ at a given day $d$. In other words, he thinks that previous 10-minute return predicts future 10-minute return, but the coefficient between them might change intraday.

Of course, to fit $\beta_t$ he can use $T$ ordinary least square regressions, one for each “$t$”, but:

(a) his data set is fairly small $D$=300, $T$=100;

(b) he thinks that signal is very small, at best it has correlation with the target of 5%.

He hopes that some machine learning method that can combine regressions from nearby intraday times can help.

How would you solve this problem? Data provided is an $x$ matrix of predictors of size $300\times100$ and a $y$ matrix of targets of size $300\times100$.

Answer 1

The post is quite old, but an interesting question indeed. Here is how I would go about it:

In a regression $y_n = \alpha + \beta x_n + u_n$ the estimator of the slope coefficient is $\beta = \text{cov}(x, y)/\text{std}(y)$ To avoid distractions let's assume for a moment that everything is standardized: means are zero and variances are one. Then, the slope coefficient is just estimated by $\beta = (1/N) \sum_n x_n y_n$. I want to focus on this quantity in order to motivate the strategy.

Going back to your notation, where we have $D$ days each split into $T$ slices. In the case where each slice is viewed independently, the slope of each slice is given by $$ \beta_t = \frac{1}{D} \sum_d x_{d,t}y_{d,t} $$ This is one extreme, and can give very volatile estimates if the true dependence is small, as you point out. The noise will swamp the signal and every slice will have a very different estimate to the adjacent ones.

The other extreme is to forget about slices, and have a single estimate for all. This would be equal to $$ \beta_\infty = \frac{1}{TD} \sum_t \sum_d x_{d,t} y_{d,t} $$ Obviously this is robust but ignores the intraday variability. The question is then: is there a scheme in-between the two extremes?

If we write $\beta_\infty$ in terms of the $\beta_t$s, then we can motivate such a scheme $$ \beta_\infty = \frac{1}{T} \sum_t \left[ \frac{1}{D} \sum_d x_{d,t} y_{d,t} \right] = \frac{1}{T} \sum_t \beta_t $$ The rigid estimator is just the average of the slice estimators. And whenever you have a global average you can put instead a Kernel to average locally. This would in effect pool information from nearby slices.

If I take a Kernel function $K_h$ with bandwidth $h$, then for each time slice $t$ I can define the weights $$ w_{t,\tau} = \frac{K_h(t-\tau)}{\sum_{t'} K_h(t'-\tau)} $$

Then, my estimated slope for the slice $t$ is given by $$ \beta^h_t = \sum_\tau w^h_{t,\tau} \beta_\tau = \frac{1}{D} \sum_\tau \sum_d w^h_{t,\tau} x_{d,\tau} y_{d,\tau} $$ You can confirm that as the bandwidth goes to infinity all weights go to $1/T$ and we end up with $\beta_\infty$; as the bandwidth goes to zero the weights go to zero except for $\tau=t$ and we end up with the single-slice $\beta_t$.

This is a sketch, which can be made more precise to account for time varying volatility, means etc. I suppose using Kernels is the 'machine learning' bit.

Answer 2

As was hinted at in the comments but not stated explicitly, there are multiple issues here. I would focus on the error term. The regression as stated violated many of the conditions for least squares. There is obviously going to be autocorrelated errors and heteroscedasticity. For the second, most volatility and trading happens at the start and toward the close of the trading day, so treating these intervals as equal is not appropriate. Additionally, depending on the security, with 10 minute intervals there may be a huge number of periods with no change in price, so you also have been blessed with missing observations. Since this is a test question, and not for real, pointing out these issues show that you know your way around the assumptions and weaknesses of least squares. I also really cannot make out your notation and cannot understand how much data you really have. So, without understanding your dataset, I'll just suggest that I would break up the days into parts and estimate the periods independently and graph the betas and regression statistics over the day to look for a pattern.