After reading papers on reinforcement learning with respect to the problem of optimal execution (Nevmyvaka et al (2006), Ning et al (2018), etc), I was wondering if the Markov property assumed in all these papers is rational? Has there been at least an empirical comparison/paper detailing tradeoffs & performance of non-Markovian models with less state variables vs a Markov model with more state variables someone could direct me towards?
$\begingroup$
$\endgroup$
2
asked Dec 18, 2020 at 19:57
-
1$\begingroup$ Do you have something in particular that doesn't make sense to treat as markovian? In general, for modeling one has to make simplifications in order to derive a solution, and the Markov property makes the problem a LOT easier. Sometimes it makes sense, sometimes it doesn't depending on the problem at hand. $\endgroup$– Pontus HultkrantzCommented Dec 19, 2020 at 13:37
-
$\begingroup$ There's a sizeable class of literature/research on non-Markovian reward systems outside this problem, so I guess I'm just looking for comparison of models between for example trying to make a system Markov by adding more state variables like bid-ask spread, signed volume, etc. vs non-Markovian system w/ less state variables $\endgroup$– pSrIoGcNeAsLs - bye stackGPTCommented Dec 19, 2020 at 15:23
Add a comment
|