5
$\begingroup$

DeepMind have demonstrated amazing capabilities of a reinforcement machine learning agent to competently play Atari video games. It is most astounding that that during training nothing more than the image frames of the game and the score were provided to the deep Q-network (DQN). The agent learned appropriate actions to accurately play a game and operate competently without any specific adaptions to the source code or network hyper-parameters. It simply needs training on a large number of game sequences to learn a new game.

Could this technology feasibly be adapted to permit a machine learning agent to take competent & appropriate trading actions in the financial markets? Just like playing Pong, but with the markets? A high score would be quite agreeable!

Does anyone have experience to articulate or advice on how this could be practically experimented upon?

Improve this question
$\endgroup$
3
  • 4
    $\begingroup$ I saw the short documentary on DeepMind online. I think the reason it worked on the atari game is because the video game was predictable, the ball bounces back and forth & the machine learns to quickly position itself to avoid making the ball drop... in the markets what is to say the ball won't keep dropping! $\endgroup$
    – Rime
    Jul 22, 2016 at 7:09
  • $\begingroup$ @Rime Permit the agent to have a stop loss as one of the available actions it can take? In some of the Atari games for example there were 18 different possible actions. In the financial markets it could be reduced to three, go long, go short, go flat? $\endgroup$
    – GoFaster
    Jul 22, 2016 at 7:17
  • $\begingroup$ A warm welcome to Quant.SE and thank you for this really interesting question. You might want to have a look at my answer here: quant.stackexchange.com/a/969/12 $\endgroup$
    – vonjd
    Jul 22, 2016 at 16:24

2 Answers 2

Reset to default
4
$\begingroup$

This is an interesting question.

I would reformulate a little bit your question and try an attempt of answer of why using neural networks is not a good idea for predicting market direction.

IMHO, one main reason would be that it is not possible to experiment a strategy without modifying the market behavior and thus it is impossible to repeat the same experiment again and again, which is a prerequisite of training.

To make an analogy with the video-game machine learning, it is as if when you start the level again, the traps where you have lost have been replaced by new ones you never thought about it before.

I encourage other quant.stackexchange members to share their view on the subject.

Improve this answer
$\endgroup$
3
  • $\begingroup$ So you think it should be trained on different periods every time - so it cannot learn to /remember/ the future, it only has the past to go off? Something i've always found interesting is the idea that everything is markovian - if that were the case, then how can the past be a useful tool for training :p. $\endgroup$
    – will
    Jul 22, 2016 at 8:20
  • $\begingroup$ Markov is ok, because it means there is a kind of global transition matrix which is always the same. What I wanted to say was that logically speaking it is wrong to say you backtested your strategy on the past with markets because the past would have been different with you being market participant applying your strategy. $\endgroup$
    – M. Jeunesse
    Jul 22, 2016 at 12:28
  • $\begingroup$ Yes, that's part of the approximation - it depends on the size of the trades your strategy is meant to do - you have to model what you're looking to take advantage of; if you want to make a model that executes large trades efficiently, obviously you have to take into account your influence on the order book, if you're only looking at small trades, or otc stuff, then it's not going to matter as much. $\endgroup$
    – will
    Jul 22, 2016 at 12:52
3
$\begingroup$

Z. Jiang, D. Xu, J. Liang, in A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem. demonstrate a Deep RL framework for Trading. The approach is based on Tensor flow and uses the ideas similar to the Open AI Gym used by Deepmind for video games!

In my blog Optimizing a Portfolio of Cryptocurrencies with Deep Reinforcement Learning I pick up their framework and see how the same strategy performs after the crypto crash.

Please have a look!

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.