In Linear Regression for time series stock prediction, instead of using the cost function and minimizing the cost function, why can't we use the final portfolio value? Assume we are doing a time series prediction, where each day we are either long or short a particular stock or index, and never in cash (this could be a logistic regression but let's use linear regression).

For example, start with $10k, and run regression using 5 years of data. We are doing a linear regression on daily returns (or we could cube root the returns to transform them to reduce the affect of crashes, etc, on the data, that are caused by factors that are probably not in the features used as predictors).

In any case, we minimize the cost of each day's prediction. For example, assume the prediction predicts +0.030, then we would be long, if it predicts -0.001 (slightly negative), then we should be short, always the entire portfolio value. We minimize the cost function which is just the error, (H(x)-y)^2.

Instead, we can take each day's prediction, and the actual outcome had we followed that prediction, and starting with say, $10k, simulate what the final portfolio value would be (we can add slippage for commission, etc, but I don't think that would matter here). Then, instead of minimizing the cost function, we maximize the final portfolio value.

What are the pros/cons of doing it the second way? I know we wouldn't be able to use gradient descent, for one thing. Also it is probably more susceptible to overfitting, but the transform can/should help with that, right? By transform, I mean say system predicted "0.040" which is a buy or long. The actual "cost" is that by being long, we gained 9.0%, which is out of the ordinary (some news came out, earnings, etc, that system doesn't predict or can't predict, and we don't want to "over-fit" this). By doing a cube root transform, we use 0.09^(1/3)=0.448. We can divide by 100 to avoid too high of returns, and assume today's return was 0.448% instead of 9%. We would tabulate final portfolio value using the transformed returns from our predictions, instead of actual returns. This has the affect of making outliers (like the 9% gain in one day) have a lower affect on the "cost".

So, what are the reasons we don't use final portfolio value maximization as the "cost function", instead of the normal 12m∑i=1m(hθ(X(i))−Y(i))2?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.