Least Median Squares is often argued to give more stable results than does OLS. Whereas in OLS one minimises the mean of squared residuals, in LMS, one instead minimises the median of squared residuals. Intuitively, should give estimators that are largely (completely?) invariant to outliers. As such, i would have thought this approach would find a natural home in finance applications, however, I haven't come across it being used (a search on this site, for example gives zero citations).

Does any one recommend or discourage a LMS approach - say, for a simple case of estimating a beta coefficient between two securities?

  • 2
    $\begingroup$ I know one reason that mean squared error rather than median squares. Has lately to do with the fact that the estimates of coefficients of regression can be easily found via linear algebra (since mean squared error is essentially has relation to norm linear algebra so can use idea of projections) this method is very versatile as it takes away the tedious manner of estimating parameters through a calculus minimization method $\endgroup$ – Kamster Jan 22 '15 at 8:40
  • 3
    $\begingroup$ @Kasmter is spot on. Most of it is just speed. With least squares, many common problems we face (e.g., curve fitting w/ cubic splines) becomes linear regression, with is really fast and easy to solve. But with least median, we need to use numerical optimizers, which is slower and could have global convergence issues. $\endgroup$ – Helin Jan 23 '15 at 2:39
  • 1
    $\begingroup$ I'd understand that deficient computational power might have been a constraint to such an approach historically, but with modern day tools, surely the priority becomes one of which method gives more accurate results. $\endgroup$ – Yugmorf Jan 24 '15 at 13:55
  • $\begingroup$ Even if it is sped up, making sure that the optimizers find the best solution is non-trivial as a general question. In some cases (e.g. say if the optimization problem is convex plus a few other conditions), it could be feasible, but with full generality, optimization can be a delicate situation. $\endgroup$ – FinanceGuyThatCantCode Apr 11 '17 at 13:15

Interesting idea. I'm guessing this isn't used for two reasons:

First, the only algorithm I could find is $O(n^3)$, which is horrible if you're using a moderately-sized high-frequency dataset. Least squares is $O(nk^2)$ (n is the number of rows, and k is the number of predictors; typically $k<<n$).

More relevantly, L1 regression is almost as outlier-insensitive as LMS, and that can be solved easily and quickly with any LP-solver. For the kinds of outliers we see in finance, I doubt there would be a material difference in the L1 and LMS solutions.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.