2
$\begingroup$

I'm trying to predict the one-day ahead movement of the S&P 500 with Temporal Convolutional Networks 1 to capture some "memory".

TCN architecture

I use daily close data with the loss function $\mathrm{MSE}(f(x_0, \ldots, x_T), y_T)$ where $f(x_0, \ldots, x_T) = \hat y_T := x_{T+1}$ is the output of the neural network. I've tried countless of hyperparameters with this very rudimentary model. But almost all models that do converge close enough converge to the naive estimation of the future closing price by the last known price. I've tried adding more features like VIX and interest rates to no avail.

Otherwise I'm employing early stopping, drop out for regularization, weight normalization for normalization etc and have tried long (almost a year) and short (week) input sequences.

I realize that this is a very rudimentary model and I did not expect anything ground breaking. I want to understand why things behave like they do.

enter image description here enter image description here

Question:

  1. Why is it that this network, with the bare minimum of information, convergs to the naive estimation of the last known price?
  2. How can one make small alterations, perhaps to the loss function or somthing else so it doesn't get "stuck" converging to the naive estimator?
Improve this question
$\endgroup$
11
  • 6
    $\begingroup$ Not to be too flippant, but is this question in some sense similar to, "I'm walking down a street, and I don't see \$100 bills littering the ground. How can I walk down a street and find \$100 bills everywhere?" $\endgroup$
    – Matthew Gunn
    Commented Mar 6, 2023 at 19:17
  • 1
    $\begingroup$ What's a reasonable null hypothesis? What would you expect to see here? $\endgroup$
    – Matthew Gunn
    Commented Mar 6, 2023 at 19:26
  • 1
    $\begingroup$ Something that would be kind of cool/interesting is if your technique always led to the forecast price being higher than the prior day's price by mean return of the S&P 500 over your training sample. $\endgroup$
    – Matthew Gunn
    Commented Mar 6, 2023 at 19:41
  • 3
    $\begingroup$ "maybe one could say in a way that if the end result is the last known price then the model has overfit?" – No, definitely not. The model "tomorrow's closing price will be approximately today's closing price" is extremely simple and extremely well-supported by the data; that's pretty much the exact opposite of overfitting. $\endgroup$
    – Sophie Swett
    Commented Mar 7, 2023 at 2:19
  • 1
    $\begingroup$ Keep pulling at this string... The fact that this converges on the naive guess and, crucially, why, might be an interesting paper $\endgroup$
    – ScottishTapWater
    Commented Mar 7, 2023 at 16:19

2 Answers 2

Reset to default
7
$\begingroup$

As @Bob Jansen says above the last price is actually an excellent predictor, but you should do it in the return space.

Goyal and Welch (2007) try to do this with multiple predictors and find that nothing consistently beats the naive average historical return. It would be very interesting if your model converges to the same answer. Also why do this at a daily frequency? What happens if you do monthly?

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you! Is there a reason why one should prefer the return space over the actual price? Don't returns inevitably remove some of the past memory when you take the quotient? $\endgroup$
    – Lejoon
    Commented Mar 6, 2023 at 19:35
  • 1
    $\begingroup$ Returns are stationary while prices are not! $\endgroup$
    – phdstudent
    Commented Mar 6, 2023 at 22:11
  • $\begingroup$ Might be a stupid question but why is it important that they are stationary for a model like this? The idea here was to have the temporal convolutional network decide for itself how to adress longer term memory effects. $\endgroup$
    – Lejoon
    Commented Mar 6, 2023 at 22:17
4
$\begingroup$
  1. Because the last known price is an excellent predictor.
  2. I would try modelling returns instead of prices. Now the model can't converge to last price.
Improve this answer
$\endgroup$
2
  • $\begingroup$ It can still converge to a 0 return which is in some sense equivalent to the last price in price space! Or hopefully the algo converges to SLIGHTLY different from zero in that it's the average daily return in your sample. $\endgroup$
    – Matthew Gunn
    Commented Mar 6, 2023 at 18:53
  • $\begingroup$ As I read all of the comments and answers I realize maybe the last known price should be my "null hypothesis". Could it be seen as overfitting the model? Is the setting "too general" for it to stick to anything that is not the last price, even if it's a worse predictor in the end than the last price? $\endgroup$
    – Lejoon
    Commented Mar 6, 2023 at 19:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.