On top of page 6, there is a HJB equation I don't know where it is from and how to understand it. I am new to this topic so I searched many books regarding to optimal control and HJB but still couldn't figure it out. Could you help please?
without knowing what you're trying to ask exactly:
Overall HJB is used in continuous time optimal control problems. In this paper and this equation particularly, the objective is to solve for optimal bid/offer spread given current time and your current inventory. To do this we must obtain the value function using these two parameters.
By bellman's principal of optimality, at each time step we want to minimize (or maximize when we have utility gain instead of cost function) value function update + utility loss (or gain). Value function update from time $t$ to $t+dt$ is first two items of taylor series expansion of value function;
For gain, think of at each time $t$, if your inventory didn't hit upper and lower limit $Q$, you can either sell or buy from market participants, and earn that spread accordingly. By doing each, your cash inventory, would decrease or increase by $s$, and increase by the bid/offer spread you earned. Since incoming orders on both sides have different arrival intensity (probability of occurrence), you "discount" your utility gain by the probability.
With the system set up and constraints, you can solve for optimal value function. Hope this helps.