# Dynamic programming and Bellman equation to obtain the maximum

This is the problem of Marhsall (1992) "Inflation and Asset Returns in a Monetary Economy" and Balvers and Huang (2009) "Money and the C-CAPM"

Suppose an endowment economy where the representative agent maximizes her expeced lifetime utility subject to a budget constraint and given that transaction costs to purchase consumption goods are mitigated by money holdings, that is: $$V(w_t,m_t,x_t)=\max_{c_t,\mu_t,\{s_t^i\}_{i=1}^{n}}\left(u(c_t)+\delta\mathbb{E}_{t}\left[V(w_{t+1},m_{t+1}, x_{t+1}\right)]\right) \quad (1)$$ $$\text{subject to}\quad w_{t+1}=R_{t+1}\left[w_t-c_t-T(c_t,m_t) + m_t -\mu_t\right] \quad (2)$$ $$R_{t+1}=\sum_{i=0}^{n}s_t^i R_{t+1}^{i}, \quad \sum_{i=0}^{n}s_t^i=1 \quad\text{and} \quad (3)$$ $$m_{t+1} = (\mu_t + z_{t+1})/\pi_{t+1} \quad (4)$$ where

• $$w_t,$$ real financial wealth (excluding money holdings)
• $$m_t,$$ real money holdings
• $$x_t,$$ set of state variables, exogenous to the consumer-investor, that is sufficient to represent changes in the investment opportunity set
• $$c_t,$$ current consumption
• $$\mu_t$$, real money holdings (for use in the upcoming period)
• $$\pi_{t+1}=p_{t+1}/p_t$$, gross inflation rate
• $$z_{t+1} = (M_{t+1} - M_{t})/p_t$$, transfer of government revenues from money creation
• $$T(c_t,m_t)$$ represents the real transaction cost of purchasing the current level of consumption
• so $$m_{t+1}$$ are the deflated money holdings of the next period

And it also holds for $$T(\cdot,\cdot)$$ that it is twice differentiable and also:

• $$T(c_t,m_t)\geq 0$$, $$T_{c}(c_t,m_t)> 0$$, $$T_{m}(c_t,m_t)< 0$$
• $$T_{cc}(c_t,m_t)\geq 0$$, $$T_{mm}(c_t,m_t)\geq 0$$ and $$T_{cm}(c_t,m_t)\leq 0$$

I cite the problem above, since it's been years from the last time I have a course in dynamic programming and at the moment I am struggling to solve it.

This is from the appendix of the paper. This is all about the solution of the problem and nothing else

• I mention here, that the indexes of the function $T(\cdot,\cdot)$ denote the partial derivative with respect to consumption, the money holdings and the cross derivative of consumtpion and money holdings etc. Also $\delta$ in the RHS of the value of lifetime utlility is a subjective discount factor. May 31 '20 at 22:21
• Do you have a question?
– Drew
Jun 1 '20 at 3:46
• Yes, isn't that obvious? How to solve this problem. How to take the FOC and by using the Bellman equqation to find the maximum... Jun 1 '20 at 6:33
• You will not get a closed form solution to this very complex problem. Even proving the existence of V would -- I think -- require stating additional conditions on u, mu, and some general equilibrium conditions. Perhaps you would be satisfied if someone were to help you write the Lagrangian function that can be used to derive the first-order conditions in the appendix of the paper. These conditions may be as close as the author of the paper gets to a (numerical) solution of the Bellman equation, in any case.
– Drew
Jun 1 '20 at 15:52
• Writing down the Lagrangian seems like a good first step Jun 2 '20 at 12:40