# Predict the financial markets in the fashion of a video game?

DeepMind have demonstrated amazing capabilities of a reinforcement machine learning agent to competently play Atari video games. It is most astounding that that during training nothing more than the image frames of the game and the score were provided to the deep Q-network (DQN). The agent learned appropriate actions to accurately play a game and operate competently without any specific adaptions to the source code or network hyper-parameters. It simply needs training on a large number of game sequences to learn a new game.

Could this technology feasibly be adapted to permit a machine learning agent to take competent & appropriate trading actions in the financial markets? Just like playing Pong, but with the markets? A high score would be quite agreeable!

Does anyone have experience to articulate or advice on how this could be practically experimented upon?

• I saw the short documentary on DeepMind online. I think the reason it worked on the atari game is because the video game was predictable, the ball bounces back and forth & the machine learns to quickly position itself to avoid making the ball drop... in the markets what is to say the ball won't keep dropping! – Rime Jul 22 '16 at 7:09
• @Rime Permit the agent to have a stop loss as one of the available actions it can take? In some of the Atari games for example there were 18 different possible actions. In the financial markets it could be reduced to three, go long, go short, go flat? – GoFaster Jul 22 '16 at 7:17
• A warm welcome to Quant.SE and thank you for this really interesting question. You might want to have a look at my answer here: quant.stackexchange.com/a/969/12 – vonjd Jul 22 '16 at 16:24

This is an interesting question.

I would reformulate a little bit your question and try an attempt of answer of why using neural networks is not a good idea for predicting market direction.

IMHO, one main reason would be that it is not possible to experiment a strategy without modifying the market behavior and thus it is impossible to repeat the same experiment again and again, which is a prerequisite of training.

To make an analogy with the video-game machine learning, it is as if when you start the level again, the traps where you have lost have been replaced by new ones you never thought about it before.

I encourage other quant.stackexchange members to share their view on the subject.

• So you think it should be trained on different periods every time - so it cannot learn to /remember/ the future, it only has the past to go off? Something i've always found interesting is the idea that everything is markovian - if that were the case, then how can the past be a useful tool for training :p. – will Jul 22 '16 at 8:20
• Markov is ok, because it means there is a kind of global transition matrix which is always the same. What I wanted to say was that logically speaking it is wrong to say you backtested your strategy on the past with markets because the past would have been different with you being market participant applying your strategy. – MJ73550 Jul 22 '16 at 12:28
• Yes, that's part of the approximation - it depends on the size of the trades your strategy is meant to do - you have to model what you're looking to take advantage of; if you want to make a model that executes large trades efficiently, obviously you have to take into account your influence on the order book, if you're only looking at small trades, or otc stuff, then it's not going to matter as much. – will Jul 22 '16 at 12:52

Z. Jiang, D. Xu, J. Liang, in A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem. demonstrate a Deep RL framework for Trading. The approach is based on Tensor flow and uses the ideas similar to the Open AI Gym used by Deepmind for video games!

In my blog Optimizing a Portfolio of Cryptocurrencies with Deep Reinforcement Learning I pick up their framework and see how the same strategy performs after the crypto crash.