9
$\begingroup$

If I have a simple linear regression that has statistical signification but I would like to improve the overall prediction results. Will a Kalman filter be always an improvement or as least achieve similar results to linear regression?

Edit:

Relevant Threads: How to tune Kalman filter's parameter?

$\endgroup$
6
$\begingroup$

There is no a "yes/no answer" to that question. Generally Kalman Filter tends to be better than linear regression, but everything depends on

  1. the data which you have,
  2. how you calibrate your model.

I expect that you have used some library for estimating linear regression parameters. Now you need to think how will you "tune" Kalman filter - the constants F, H, R, Q. See Wiki Page of Kalman Filter. I have asked a related question and Kalman Filter parameters tuning is not as easy as in the linear regression example.

General rule is - simple models tends to be better than complicated ones. Take a look at the quote from Makridakis Competitions.

"The most interesting test of how academic methods fare in the real world was provided by Spyros Makridakis, who spent part of his career managing competitions between forecasters who practice a "scientific method" called econometrics -- an approach that combines economic theory with statistical measurements. Simply put, he made people forecast in real life and then he judged their accuracy. This led to a series of "M-Competitions" he ran, with assistance from Michele Hibon, of which M3 was the third and most recent one, completed in 1999. Makridakis and Hibon reached the sad conclusion that "statistically sophisticated and complex methods do not necessarily provide more accurate forecasts than simpler ones.""

$\endgroup$
5
$\begingroup$

There is no magic in the Kalman Filter. The linear regression model usually assumes the coefficients follow a random walk and as such it essentially boils down to an estimation followed by exponential smoothing of the coefficients.

$\endgroup$
  • $\begingroup$ Thanks for the insights Craig. I think I understand but do you mind to elaborate on "exponential smoothing of the coefficients for random walk coefficients"? $\endgroup$ – hotsource Sep 22 '15 at 15:12
  • $\begingroup$ The Kalman Filter needs a 'model', usually in stat-arb work we are trying to estimate a regression model. If we assume the regression model has no intercept we are left with one coefficient for our model, the 'beta'. At each point the Kalman Filter estimates beta given the new information, that information is incorporated into our new estimation using the 'Kalman Gain' parameter. When you work through the equations in this case, the 'Kalman Gain' is functionally equivalent to an exponential smoother. Each new estimation is incorporated as if the estimations where being feed into an EMA. $\endgroup$ – Craig Sep 22 '15 at 20:32
1
$\begingroup$

Yes, linear regression can be cast as a Kalman filter estimate. I believe, D. Simons book "Optimal State Estimation: .. " has all the details.

$\endgroup$
  • $\begingroup$ Thanks LazyCat. The book you mentioned looks very good so I just ordered one copy. $\endgroup$ – hotsource Sep 24 '15 at 22:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.