# Hamilton-Jacobi-Bellman equation in Merton Model

I'm trying to study the Merton Model for portfolio optimization and the document doesn't explain a quite important step : if $$V(t,x)=\sup\{E[U(X_T(\phi))~|~X_t=x]~~ |~~\phi~~\text{an admissible trading strategy}\}$$ is the value function then, "under some regularity assumptions", it will satisfy the Hamilton-Jacobi-Bellman equation.

What are those regularity assumptions ? How can we prove them ?

• Can you provide the reference? Jan 31 '17 at 17:38
• If $V(t,x)$ is continuously differentiable in x and t then you should be alright. Jan 31 '17 at 19:11
• LocalVolatility, here's the reference : fields.utoronto.ca/programs/scientific/09-10/finance/courses/… Pages 18 to 21.
– mlx
Jan 31 '17 at 21:23
• I saw you follow-up questions to Quantuple's excellent answer. When studying this topic, I found the book "Theory of Asset Pricing" by George Pennacchi extremely useful and would recommend it to you for further reading. Feb 2 '17 at 13:44

This is an optimal control problem.

Consider a self-financing strategy $\pi := (\pi_s)_{s\in[t,T]}$ over the horizon $[t,T]$ consisting in, over each infinitesimal period of time $[t,t+dt[$, investing a fraction $\pi_t$ of the current wealth in a risky asset $S_t$ and placing the remaining part in the risk free asset $B_t$. Given the following dynamics $$dS_t = S_t(\mu_t dt + \sigma_t dW_t)$$ $$dB_t = B_t(r_t dt)$$ starting from an initial wealth $x$ is, the wealth at time $t$ of an investor following the strategy $\pi$ will be $$X_t^{\pi,x} = \frac{ \pi_t X_t^{\pi,x} }{ S_t } S_t + \frac{ (1-\pi_t) X_t^{\pi,x} }{B_t} B_t$$ and its evolution will be governed by the following SDE $$dX_t^{\pi,x} = X^{\pi,x} \left[ (r_t + \pi_t(\mu_t-r_t))dt + \pi_t \sigma_t dW_t \right]$$

Consider the value function $$V(t,x;(\pi_s)_{s\in[t,T]}) = \Bbb{E}_t \left[ U(X_T^{\pi,x}) \right]$$

The optimal control $\pi_t^*$ is the stochastic process such that $$(\pi^*_s)_{s \in [t,T]} = \text{argsup}_{(\pi_s)_{s \in [t,T]}} V(t,x;(\pi_s)_{s \in [t,T]})$$

while the optimal cost is $$V(t,x) = \Bbb{E}_t \left[ U(X_T^{\pi^*,x}) \right]$$

The optimal cost function solves the Hamilton-Jacobi-Bellman equations.

The proof can be obtained by viewing the control problem as a Dynamic Programming Problem and relying on Bellman's principle of optimality (see $(1)$ below).

As @noob2 mentions, at some point the Itô differential of the optimal cost $V(t,x)$ appears. Therefore regularity conditions are the usual conditions for Itô integration, both for $X_t$ and $V(t,x)$.

Some intuition \begin{align} V(t,x) &= \Bbb{E}_t \left[ U(X_T^{\pi^*,x}) \right] \\ &= \Bbb{E}_t \left[ \Bbb{E}_{t+dt} \left[ U\left(X_T^{\pi^*,x+dX_t(\pi^*_t)}\right) \right] \right] \\ &= \Bbb{E}_t \left[ V(t+dt, x+dX_t(\pi^*_t)) \right] \tag{1}\\ &= \sup_{\pi_t} \Bbb{E}_t \left[ V(t+dt, x+dX_t(\pi_t)) \right]\\ &= \sup_{\pi_t} \Bbb{E}_t \left[ V(t,x) + \frac{dV(t,x)}{dt} dt + \frac{dV(t,x)}{dx}dX_t + \frac{d^2V(t,x)}{dx^2}d\langle X \rangle_t \right] \\ &= V(t,x) + \frac{dV(t,x)}{dt} dt + \sup_{\pi_t} \left( \frac{dV(t,x)}{dx} x (r_t + \pi_t(\mu_t-r_t)) + \frac{1}{2} \frac{d^2V(t,x)}{dx^2} x^2 \pi_t^2 \sigma_t^2 \right) dt \end{align} hence finally $$\frac{dV(t,x)}{dt} + \sup_{\pi_t} \left( \frac{dV(t,x)}{dx} x (r_t + \pi_t(\mu_t-r_t)) + \frac{1}{2} \frac{d^2V(t,x)}{dx^2} x^2 \pi_t^2 \sigma_t^2 \right) = 0$$

The DPP point of view consists in viewing the optimal control $(\pi^*_s)_{s \in [t,T]}$ as the "union" of what you choose to do over $[t,t+dt[$ and what you do over $[t+dt,T[$. Informally: $$(\pi^*_s)_{s \in [t,T]} = \pi^*_t \cup (\pi^*_s)_{s \in [t+dt,T]}$$

At this point, Bellman's optimality principle tells you that the restriction of the optimal control $(\pi^*_s)_{s \in [t+dt,T]}$ is itself the optimal policy over the horizon $[t+dt,T[$. This is why in $(1)$ you can write that $$\Bbb{E}_{t+dt} \left[ U\left(X_T^{\pi^*,x+dX_t(\pi^*_t)}\right) \right] = V(t+dt,x+dX_t(\pi^*_t))$$ with $V$ the optimal cost (and not simply the value function).

• Thank you for your response. I have some trouble though. If I get it right, you say that $\Bbb{E}_{t+dt} \left[ U\left(X_T^{\pi,x+dX_t(\pi_t)}\right) \right] \\$ is equal to $V(t+dt,x+dX_t(\pi_t))$ which is, by your definition, the optimal cost. Shouldn't $\Bbb{E}_{t+dt} \left[ U\left(X_T^{\pi,x+dX_t(\pi_t)}\right) \right] \\$ be equal to the value fonction $V(t,x,(\pi_s)_{s \in [t,t+dt[})$ instead ? Thank you !
– mlx
Feb 1 '17 at 12:44
• @Mhamed Jabri - I've edited my answer and detailed some intermediate steps. Hopefully it is clearer for you now. Feb 1 '17 at 13:06
• Thank you very much for you last editing, it's a lot clearer to me and I get where I was wrong. I have only two last questions : 1/ Can you please give more explanation about this equality : $\Bbb{E}_t \left[ U(X_T^{\pi^*,x}) \right] = \Bbb{E}_t \left[ \Bbb{E}_{t+dt} \left[ U\left(X_T^{\pi^*,x+dX_t(\pi^*_t)}\right) \right] \right] \\$ \\ 2/ If we have $U(x)=ln(x)$ or $U(x)=x^{\lambda}~~\text{for$\lambda$in [0,1]}$, how can we go from the last equation to the obtain the solution $(\pi_t)$ ? Do you have any reference where I can see the resolution for those cases ? Thank you !
– mlx
Feb 1 '17 at 14:10
• @Mhamed Jabri Regarding your last point, it would be best to ask this as a separate question to comply with the site's single Q&A format. The equality you refer to is simply some form of tower rule for conditional expectation. $$\Bbb{E}_t \left[ U(X_T^{\pi^*,x}) \right] = \Bbb{E}_t \left[ \Bbb{E}_{t+dt} \left[ U(X_T^{\pi^*,x}) \right] \right]$$ Now, standing at $t+dt$, conditionally on $X_t=x$ you now have a wealth $X_{t+dt} = x + dX_t(\pi^*_t)$ where the control $\pi^*_t$ was applied to move from $X_t=x$ a.s. to $X_{t+dt}$. Feb 1 '17 at 14:29