3
$\begingroup$

In brief, what are some mainstream and recent applications of reinforcement learning in finance that fall outside of the usual scope of agent-based modeling?

$\endgroup$

3 Answers 3

5
$\begingroup$

Really recommend this book for RL in finance :

He talks about QLBS, q-learning setup for black scholes, RL for investment management and inverse RL for trading.

$\endgroup$
3
$\begingroup$

See Alexandr Honchar's post on portfolio optimization with RL: https://medium.com/swlh/ai-for-portfolio-management-from-markowitz-to-reinforcement-learning-cffedcbba566

$\endgroup$
1
  • $\begingroup$ Can you explain how Q-learning portfolios described in your link work? How does RL determine which action for which state, and is this restricted to day-trading $\endgroup$
    – develarist
    Commented Nov 7, 2020 at 0:29
2
$\begingroup$

I second the previous answer by Igor Rivin. In quantitative finance contexts Reinforcement Learning (RL) is chiefly employed to derive automated Portfolio Allocation Strategies showing superior performances (especially when the analyses are theoretical, and do not consider transaction costs, freely reallocating way too frequently) with respect to classical techniques such as Markowitz' MVO, Black-Litterman, Minimum Variance.

See this review paper from 2018, in which RL algorithms are classified in three distinct camps:

  • Critic-only approach: select a value function to maximize that basically cares as much for exploration than exploitation when called upon to decide for action. Choice of action happens after having detected the current state of the system/environment (the market). The decision will be based on past rewards;
  • Actor-only approach: more greedy, prefers exploitation;
  • Actor-critic approach: "The key idea is to simultaneously use an actor, which determines the agent’s action given the current state of the environment, and a critic, which judges the selected action. Simply speaking, the actor learns to choose the action which is considered best by the critic and the critic learns to improve its judgment."

On the topic of Q-learning, it pertains to what kind of object function the RL agent intends to optimize. It is implemented with Neural Networks ("Deep RL"); the whole concept of it is linked to the Bellman equations, for which I would refer readers to this clear and informative video lecture. Q-learning consists in a family of algorithm that emphasize the optimization of the value function first and foremost, and from this then they derive an optimal policy. The opposite trend is that of policy-based methods, which directly optimize the objective function for the policy (usually cumulative rewards) using numerical techniques such as Gradient Descent-based algorithms.

Here is an available report, good to get an idea for a Python and a C++ implementation of a RL-driven portfolio allocation strategy.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.