Stochastic control (HJB) for wealth process involving stopping times

Question

Given a wealth process that evolves as $$d w_t = r w_t dt + \theta_t ( \sigma dW_t + (\mu-r) dt) - c_t dt.$$ where $\theta_t$ is the worth of holding at time $t$ and $c_t$ is the consumption stream.

Also, we define smooth functions $u,F: [0, +\infty) \rightarrow \mathbb{R}$.

How can we optimise the following:

$$V(w) = \sup_{c \geq 0, \, \theta, \, \tau} \mathbb{E} \bigg[ \int_0^{\tau} e^{- \rho t} u (c_t) dt + e^{-\rho \tau} F(w_{\tau}) \bigg| w_0 =w \bigg],$$ where $\tau$ is a stopping time.

The traditional method of using the HJB and martingale principle of optimal control does not seem to work in this case, when stopping time is involved.

Any suggestions on how to optimise this?

Answer 1

This is a standard combined optimal stopping and optimal stochastic control problem.

You are looking for a control $u=(\theta,c)$ and stopping time $\tau$ which maximize the pefrormance functional of the form:

$$J^{(u,\tau)}(w):=\mathbb{E}^{w} \bigg[ \int_0^{\tau} e^{- \rho t} u (c_t) dt + e^{-\rho \tau} F(w_{\tau})\chi_{\{\tau<+\infty\}}\bigg]$$

I will not write down the whole theory (HJB verification theorem is rather long) but everything what you need you can find in this book (in even more general form, I mean when dynamics of $w_{t}$ includes jumps. For your problem just ignore the jumps in the below book.):

Authors: Bernt Oksendal, Agnes Sulem

Title: Applied Stochastic Control of Jump Diffusions

Chapter: 4. Combined Optimal Stopping and Stochastic Control of Jump Diffusions

Page: 65

Second Edition