# Closing prices are predicted very well but returns are predicted poorly

I'm learning some time series analysis and forecasting techniques, I've tried to predict stock prices for Netflix but I'm very confused.

At first I've tried Auto ARIMA which gave me a straight line, obviously it's a bad fit, then I tried a linear regression between X(t) and it's lagged version, I've plotted a lag plot and saw that there is a very strong correlation between X(t) up to X(t-10) so I trained a linear regression model using X(t-1)...X(t-6) as features (predictors) and X(t) as a target.

I've compared the predictions next to the test set and the results were quite shocking, the model was nearly perfect and predictions were almost equal to actual values in the data set.

The MAE is only 6.25 (6.25 dollars off in average).

Next I tried another ML technique which is the Gradient Boosting Trees algorithm and results were as perfect as the linear regression model, you can see the results here

So I was thinking that something was wrong and I tried changing my variable, this time instead of using closing prices I used returns (using both algorithms) and the results were very bad and very off as you can see here:

and this is when I multiply predictions by 10:

These results are very confusing for me, I'm wondering why am I fitting the closing prices almost perfectly while returns are modeled quite badly ? and most importantly What's the recommended approach to predict stock prices ?

Note: I already know that returns are stationary while closing prices tend to not be, but is this important ? and If so why ?

Thank you !

A stock price in theory is a random walk: the expected value of today’s price is yesterday’s price. So from looking at a chart, yes a regression of $$S_t$$ on $$S_{t-1}$$ will appear very close in terms of fit. That’s because the price today is that of yesterday plus or minus a random element centered around zero.