After reading papers on reinforcement learning with respect to the problem of optimal execution (Nevmyvaka et al (2006), Ning et al (2018), etc), I was wondering if the Markov property assumed in all these papers is rational? Has there been at least an empirical comparison/paper detailing tradeoffs & performance of non-Markovian models with less state variables vs a Markov model with more state variables someone could direct me towards?

  • 1
    $\begingroup$ Do you have something in particular that doesn't make sense to treat as markovian? In general, for modeling one has to make simplifications in order to derive a solution, and the Markov property makes the problem a LOT easier. Sometimes it makes sense, sometimes it doesn't depending on the problem at hand. $\endgroup$ – Pontus Hultkrantz Dec 19 '20 at 13:37
  • $\begingroup$ There's a sizeable class of literature/research on non-Markovian reward systems outside this problem, so I guess I'm just looking for comparison of models between for example trying to make a system Markov by adding more state variables like bid-ask spread, signed volume, etc. vs non-Markovian system w/ less state variables $\endgroup$ – pSrIoGcNeAsLs Dec 19 '20 at 15:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.