1
$\begingroup$

I am trying to derive the HJB equation in a stochastic setting. Let me exemplify my problem with the simplest case where there is no control, just one state variable. Assume the payoff is given by $$ V(X_{t})\equiv E_{t}\left\{ \int_{t}^{\infty}e^{-\rho(s-t)}u(X_{s})ds\right\} $$ where $X_{t}$ is given by $$ dX_{t}=\mu(X_{t})dt+\sigma(X_{t})dZ_{t} $$ and $Z_{t}$ is the standard Brownian Motion. For any $dt>0$ we can write: $$ V(X_{t})=E_{t}\left\{ \int_{t}^{t+dt}e^{-\rho(s-t)}u(X_{s})ds+e^{-\rho dt}V(X_{t+dt})\right\} $$

$$ \left(1-e^{-\rho dt}\right)V(X_{t})=E_{t}\left\{ \int_{t}^{t+dt}e^{-\rho(s-t)}u(X_{s})ds+e^{-\rho dt}\left[V(X_{t+dt})-V(X_{t})\right]\right\} \tag{1} $$

From Ito calculus we get that (and assuming that $W(\cdot)$ is well behaved): $$ V(X_{t+dt})-V(X_{t})=\int_{t}^{t+dt}V'(X_{s})dX_{s}+\frac{1}{2}\int_{t}^{t+dt}V''(X_{s})d[X_{s}]=\int_{t}^{t+dt}V'(X_{s})dX_{s}+\frac{1}{2}\int_{t}^{t+dt}\sigma(X_{s})^2V''(X_{s})ds $$ where the last equality follows from the known properties of the quadratic variation of the process $X_{t}$. Plugging this back in (1): $$ \left(1-e^{-\rho dt}\right)V(X_{t})=E_{t}\left\{ \int_{t}^{t+dt}e^{-\rho(s-t)}u(X_{s})ds+e^{-\rho dt}\left[\int_{t}^{t+dt}V'(X_{s})dX_{s}+\frac{1}{2}\int_{t}^{t+dt}\sigma(X_{t})^2V''(X_{s})ds\right]\right\} $$

Dividing both sides by $dt$ and taking the limit $dt\rightarrow0$: $$ \rho V(X_{t})=E_{t}\left\{ u(X_{t})+\lim_{dt\rightarrow0}\frac{\int_{t}^{t+dt}V'(X_{s})dX_{s}}{dt}+\frac{1}{2}\sigma(X_{t})^2V''(X_{t})\right\} $$ where I used the fact that when dealing with the Riemann integral: $\lim_{dt\rightarrow0}\frac{\int_{t}^{t+dt}f(x_{s})ds}{dt}=f(x_{t})$ (from standard calculus).

As you can see, I am almost there. I just don't know how to deal with term $\lim_{dt\rightarrow0}\frac{\int_{t}^{t+dt}V'(X_{s})dX_{s}}{dt}$. For example, assume $\mu(X_{t})=0$ and $\sigma(X_{t})=1$, so that $X_{t}$ is simply the standard Brownian Motion $Z_{t}$. In that case, to get the HJB formula right I would need: $$ \lim_{dt\rightarrow0}\frac{\int_{t}^{t+dt}V'(Z_{s})dZ_{s}}{dt}=0 $$ But I don't know how to prove that this is true. More generally (for any $\mu(X_{t})$ and $\sigma(X_{t})$), I would need to prove: $$ \lim_{dt\rightarrow0}\frac{\int_{t}^{t+dt}V'(X_{s})dX_{s}}{dt}=\mu(X_{t})V'(X_{t}) $$ which I am also not sure how to do. Any ideas?

$\endgroup$
  • 1
    $\begingroup$ I am not sure to understand the point of the deriving the equation if there is no control, Also good notations help. Writing $W$ for you payoff is not a good choice. And your payoff is a function of time and state so $v(t,x)$ would be much better than $W(X_t)$. To answer your question simply replace $dX_t = \mu dt + \sigma dZ_t$, the expected value of the brownian part is $0$. PS: you forgot to square the volatility in the quadratic term. $\endgroup$ – AFK May 21 '15 at 19:32
  • $\begingroup$ I changed the notation. I also removed the dependence of the drift and variance on time, so that I don't need to make my value function depend on time. In some economics applications it is useful to write the value function as a differential equation. Even if it is only to get some economic intuition from HJB. $\endgroup$ – Pcw. May 21 '15 at 19:59
  • $\begingroup$ You need only note that \begin{align*} E_{t}\bigg( \lim_{dt\rightarrow0}\frac{\int_{t}^{t+dt}V'(X_{s})dX_{s}}{dt}\bigg) &= \lim_{dt\rightarrow0}E_{t}\bigg(\frac{\int_{t}^{t+dt}V'(X_{s})dX_{s}}{dt}\bigg)\\ &=\lim_{dt\rightarrow0}E_{t}\bigg(\frac{\int_{t}^{t+dt}V'(X_{s})u(X_{s})dt + \int_{t}^{t+dt}V'(X_{s})\sigma(X_{s})dZ_s}{dt}\bigg)\\ &= \lim_{dt\rightarrow0}E_{t}\bigg(\frac{\int_{t}^{t+dt}V'(X_{s})u(X_{s})dt }{dt}\bigg)\\ &=V'(X_{t})u(X_{t}), \end{align*} as pointed out by AFK. $\endgroup$ – Gordon May 21 '15 at 20:26
1
$\begingroup$

By using the fact that the brownian integral has expected value $0$, we find \begin{eqnarray*} & & \left(1-e^{-\rho dt}\right)v(X_{t}) \\ &=& E_{t}\left\{ \int_{t}^{t+dt}e^{-\rho(s-t)}u(X_{s})ds+e^{-\rho dt}\left[\int_{t}^{t+dt}v'(X_{s})dX_{s}+\frac{1}{2}\int_{t}^{t+dt}\sigma(X_{s},s)^2 v''(X_{s})ds\right]\right\} \\ &= & E_{t}\left\{ \int_{t}^{t+dt}e^{-\rho(s-t)}u(X_{s}) +e^{-\rho dt}\left(v'(X_{s})\mu(s,X_s) +\frac{1}{2}\sigma(X_{s},s)^2 v''(X_{s})\right)ds\right\} \end{eqnarray*} So the PDE is \begin{eqnarray*} \rho v(x) &=& u(x) + \left(\mu(t,x) v'(x) +\frac{1}{2}\sigma(t,x)^2 v''(x)\right) \end{eqnarray*} or \begin{eqnarray*} (- \mathcal{A}_t + \rho) v(x) &=& u(x) \end{eqnarray*} where $\mathcal{A}_t = \mu(t,x)\partial_x + \frac{1}{2}\sigma(t,x)^2 \partial_x^2$ is the generator of the diffusion.

$\endgroup$
  • 1
    $\begingroup$ You are saying that $E_t\{\int_t^{t+dt} v'(x_s)dZ_s\}=0$? I've just started to deal with continuous time stuff, but it is not obvious to to me why (I get that $E_t\{\int_t^{t+dt}dZ_s\}=0$ though). $\endgroup$ – Pcw. May 21 '15 at 20:38
  • 1
    $\begingroup$ You need integrability conditions on $v^{'}$. It must be adaptable and belong to ${{\cal L}^2}$ then you have an Ito integral whose expectation must be zero. $\endgroup$ – Rohit Arora Jun 21 '15 at 18:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.