non-Markovian/path-dependent optimal log utility and HJB-PDE

Question

Basic question:

Can we generalize the HJB PDE to apply to optimal controls of non-Markovian/path-dependent SDEs? Specifically, how do we generalize the log-optimal portfolio to path-dependent processes?

Skip down below to the Main-Question/Goal for more details if you are familiar with this background of stochastic control for classic SDEs.

Brief Setup and Review: Stochastic control problems come up often in finance. A favorite of mine is the portfolio optimizing log-utility of wealth. In the case of a market consisting of a stock $(S_t)_{t\geq 0}$ and a bond earning rate $r>0$, if the stock follows a general GBM $$dS_t = \mu(t, S_t) S_t dt+\sigma(t, S_t) dB_t,$$ i.e. where the coefficients depend on time and the current price only, then the solution is easily derived by solving the HJB-PDE for the controlled-wealth process $X$: $$dX^\alpha = [r+(\mu-r)\alpha]X^\alpha dt+\alpha \sigma X^\alpha dB_t.$$ (Here I have suppressed dependencies on $t$ and $S_t$ for brevity and the superscript $\alpha$ is not a power). Then the optimal value function $$v(t,x,s)= \max_\alpha \mathbb{E}(\log X_T |X_t=x, S_t=s),$$ must solve the HJB-PDE $$v_t+\sup_a \mathscr{L}_x^a v=0,$$ with terminal condition $v(T,x,s)=\log x$, and for each control $a$, $\mathscr{L}_x^a$ is the infinitesimal generator of the controlled process $X$. In this case, everything can be computed explicitly as follows.

Solution for Markovian GBMs After one computes the infinitesimal generator, we obtain the non-linear parabolic PDE by maximizing for the optimal control in feedback form (i.e. it depends on derivatives of the value function $v$) $$v_t+rx v_x+\mu(t, s)s v_s+\frac12 \sigma^2(t, s) s^2 v_{ss} -\frac{(\lambda(t, s) v_x+\sigma(t, s) s v_{xs})^2}{2 v_{xx}}=0,$$ for $(t, x, s) \in [0, T)\times (0, \infty) \times (0, \infty)$, and $v(T, x, s)=\log x,$ for $(x,s)\in (0,\infty) \times (0,\infty)$. Here subscripts denote partial derivatives and $$\lambda(t, s)=\frac{\mu(t, s)-r}{\sigma(t, s)},$$ is the Sharpe-ratio. Guessing $v(t,x,s)=\log x+u(t,s)$ leads us to conclude that $$v(t, x, s)=\log x +r(T-t)+\tfrac 12 \mathbb{E}^{t, s} \left[\int_t^T \lambda(w, S_w)^2 dw \right],$$ solves our original non-linear PDE in $v$. This also gives us the optimal control $$a^*(t, x, s)=\frac{\mu(t, s)-r}{\sigma(t, s)^2},$$ a clear generalization of the Kelly-fraction for the constant case $\frac{\mu-r}{\sigma^2}$.

Main Question/Goal:

An even more realistic model would incorporate path-dependency into the coefficients $\mu$ and $\sigma$. For example, we would like to have $\mu$ and $\sigma$ depend on the path of $S$ up to time $t$. Now I know some basics. For example, Bruno Dupire introduced Functional Ito calculus. A wonderful result of his is that the new functional Ito formula looks exactly the same as the standard Ito formula, except that derivatives are replaced with generalized derivatives for functionals. Because of this, the infinitesimal generator also looks the same, just with different notations of derivatives. His main application is for pricing path-dependent options. Has anyone used this functional calculus to generalize the HJB-PDE from Markovian SDEs to non-Markovian SDEs? Can we simply write down the HJB-PDE with his new notions of derivatives and obtain a formal analog of the previous case? In this case, I believe we might have a stochastic-PDE and our solutions are probably going to have to be taken in the viscosity sense. An intriguing case would be when $\mu$ and $\sigma$ are parameterized or approximated by (recurrent) neural networks, in some sense. There are a few papers I found that involve path-dependent controls, and Oskendal treats the control non-Markovian jump diffusions using FBSDEs. If I gain enough understanding of their work I will specialize it to when no jumps are present and try to post an answer but I am interested in other resources as well.

I apologize for the length of this question and the bit of rambling near the end. I hope this is not too mathematical, because while I phrased this in somewhat general terms, I am mainly interested in the non-Markovian/path-dependent generalizations of optimal log utility.

Answer 1

"Has anyone used this functional calculus to generalize the HJB-PDE from Markovian SDEs to non-Markovian SDEs? Can we simply write down the HJB-PDE with his new notions of derivatives and obtain a formal analog of the previous case?" Yes this was done by Rama Cont and David Fournie: see http://rama.cont.perso.math.cnrs.fr/pdf/BallyCaramellinoCont.pdf

Answer 2

This answer will provide somewhat of an educated guess, but is by no means rigorous or exact. It is based on my preliminary readings/understanding of subsections 5.4.1 and 5.4.2 of Applied Stochastic Control of Jump Diffusions by Oskendal and Sulem. These sections deal with optimal control of FBSDES and utility maximization, respectively. The SDE in the latter topic assumes the coefficients $\mu(t,\omega)$ and $\sigma(t,\omega)$ are adapted processes on $[0,\infty)\times \Omega$, which is, I believe, more general than the OP.

Consider the backward stochastic partial differential equation (BSPDE), for processes $y(t,x, s, s_.)$ and $z(t,x,s,s_.)$: $$dy(t,x,s,s.) =-\mathscr{L}_s y(t,x, s, s_.)dt+ C(t,x, s, s_.)dt + z(t,x,s, s_.) dB_t$$ with terminal condition $y(T,x,s, s_.)=\log x$, where $$C(t,x,s,s_.) = \frac{(y'(t,x, s, s_.) \mu(t, s_.)+\sigma(t,s_.)s \partial_{xs}y(t, x, s, s_.)+z'(t,x, s_.)\sigma(t,s_.))^2}{2y''(t,x, s, s_.)\sigma^2(t, s_.)}$$

Here $y'$ and $y''$ denote the first and second partial derivatives with respect to $x$, while $\partial_{xs}y$ is the mixed partial of $x$ and $s$ for $y$. Here, we have written $$\mathscr{L}_s y(t,x,s,s_.) = \mu(t, s_.) \partial_s y+ \frac12 \sigma(t, s_.)^2 \partial_{ss} y$$

When $z=0$ and $\sigma(t,s_.)=\sigma(t,s)$ and $\mu(t,s_.)=\mu(t,s)$ we recover exactly the above HJB PDE in the OP. This is good! The solution of this BSPDE is beyond me at the moment, but I would expect we might get something nicer by a substitution of the form $y(t,x,s,s_.) = \log(x)+u(t, s, s_.)$, just as before. If I have the energy later I will try to see if anything more can be said...

In some sense we should have, I believe, $$y(t,x, s, S_.) = Y_{t,x,s} = \operatorname{ess sup}_{\alpha} \mathbb{E}(\log(X_T^\alpha)|\mathscr{F}_t^S),$$ where we assume $S_t=s$ and $X_t=x$.

As I said on the onset, this is just merely speculation, applying the principle of mutatis mutandis from the Markovian case and some literature I've been reading—somewhat recklessly, I'll admit! The non-Markovian cases are rather to difficult to handle, and there is a lot of new machinery: BSDEs, SPDEs, BSPDEs, etc. If anyone more knowledgeable comes along and sees this, I would be grateful to hear your comments, corrections, or clarifications.