Combining BHHH and Levenberg Marquardt

Question

I already asked a question related to this here:

How to apply Levenberg Marquardt to Max Likelihood Estimation

I know understand how Levenberg Marquardt (LM) can be applied to the objective function.

In the paper on p315:

http://www.ssc.upenn.edu/~fdiebold/papers/paper55/DRAfinal.pdf

the authors also state that:

"We maximize the likelihood by iterating the Marquardt and Berndt–Hall–Hall–Hausman algorithms, using numerical derivatives, optimal stepsize, and a convergence criterion of 10^-6 for the change in the norm of the parameter vector from one iteration to the next."

Does anyone know what precisely they mean by this?

Do you run the LM fully and then take the optimal parameter set found and use those as initial guesses for the BHHH algorithm, then run the BHHH and take those solutions as initial guesses to the LM and repeat until convergence?

Or are they iterating step by step? So you do the same as above but rather than running a full optimization each time, you simply so one step at a time?

Even your best guesses would be most welcome.

Baz

Answer 1

Well, given that either LM or BHHH is supposed to stop when the Kuhn-Tucker condition is satisfied, I infer it has to be stepwise. I would say otherwise if, say, they were potentially using something like SALO (simulated annealing with local optimization), where one algorithm could profitably run in full as a sub-step of the other.