4
$\begingroup$

Hi: Can anyone recommend an introductory book on stochastic programming ? There are obviously so many books on Amazon but I can't tell easily which ones could be useful. It would be good if it had some balance between theory and application. Thanks.

$\endgroup$
  • 1
    $\begingroup$ There is a common thread I think for all book recommendations. If someone contributes here we'll try and add it there and then close this question $\endgroup$ – Attack68 Feb 11 at 17:15
  • $\begingroup$ @Attack68: Thanks but where is that common thread ? I'd like to take a look at it. $\endgroup$ – mark leeds Feb 11 at 17:27
  • $\begingroup$ I tried to find it - Its really extensive and I have added to it in the past. Hopefully someone can help me out and then Ill try to pin it somehow $\endgroup$ – Attack68 Feb 11 at 17:58
  • 1
    $\begingroup$ A few years ago W. Ziemba wrote a monograph on asset liability management by stochastic programming, perhaps it can serve as a starting point cfainstitute.org/-/media/documents/book/rf-publication/2003/… $\endgroup$ – noob2 Feb 11 at 18:26
  • 1
    $\begingroup$ @markleeds here it is.. quant.stackexchange.com/questions/38862/… $\endgroup$ – Attack68 Feb 12 at 7:45
1
$\begingroup$

I think you will want a few books since the best book for stochastic programming (but not dynamic, i.e. across time) is different than the best book(s) for stochastic dynamic programming.

For stochastic programming, Birge and Louveaux's Introduction to Stochastic Programming 2nd Ed. is the book I found most helpful. It covers many iterative and approximation techniques. It hurts me to say this (since Birge is a very good human), but I would not get the first edition: it has serious flaws with formatting in a few places. So make sure to get the 2nd edition.

For stochastic dynamic programming, Puterman's Markov Decision Processes is outstanding and even has enough theory to cover some continuous-time results. The jumping off point is stochastic processes, which I found very helpful and intuitive. I'm not sure, though, if it has as much on applications as the other two books I mention here.

You should also read up on approximate dynamic programing since that often lets you relax or reframe a stochastic problem enough to solve it more efficiently. We just read papers on the topic, but since then Powell has written Approximate Dynamic Programming which appears to be very good.

| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Thanks very much for the great explanation and information. I find it quite confusing because of the terminology: stochastic programming and stochastic dynamic programming. They're definitely different beasts. Is it correct to say that stochastic programming is the stochastic version of linear programming ( dantzig ) in operations research and stochastic dynamic programming is more like reinforcement learning. Thanks a lot. I've seen puterman recommended before in other contexts. $\endgroup$ – mark leeds Aug 19 at 12:59
  • $\begingroup$ Ah, yes: that's largely correct. Stochastic programming usually covers stochastic linear and nonlinear programming and sometimes (as in Birge and Louveaux) stochastic integer programming. Dynamic programming often gets covered separately since the time factor leads to different solution methods. Birge and Louveaux has a little on stochastic dynamic programming, but only enough to hint at the area. Not sure how close SDP is to reinforcement learning -- though they do not seem far off from what I know of RL. $\endgroup$ – kurtosis Aug 19 at 13:13
  • 1
    $\begingroup$ okay. got it. I think my next book, whenever that is, will be puterman. It has really good reviews on Amazon and RL is always being talked about on the blogs as the latest and greatest. Unfortunately, the texts are always so expensive. All the best and thanks for clarification of stochastic programming and stochastic dynamic programming !!!!! $\endgroup$ – mark leeds Aug 19 at 17:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.