44

Because of: The (extreme) dominance of noise over signal The prevalence of non-repeating patterns (many of which we know are not going to repeat) A pathetic sample size for cross-validation Regime changes due to exogenous events. These are typically in the cross-val window which makes it even worse. (GFC, financial integration, trade law changes, interest ...


24

Two aspects of statistical learning are useful for trading 1. First the ones mentioned earlier: some statistical methods focused on working on live datasets. It means that you know you are observing only a sample of data and you want to extrapolate. You thus have to deal with in sample and out of sample issues, overfitting and so on... From this viewpoint, ...


24

First, let's speak about perceptrons in general: their input $X_0$ is a $K$-dimensional vector. So if you want to use $(P_{bid}(t),P_{ask}(t), Q_{bid}(t),Q_{ask}(t))$, it would mean that without any effort (but later we will see that is would be better to do some efforts, as usual): $$X_0(t)=(P_{bid}(t),P_{ask}(t), Q_{bid}(t),Q_{ask}(t))'\in\mathbb{R}^4$$ ...


17

Stanford University has a free online course in machine learning with video lectures, problem sets, and even a promise of online help with coursework from Stanford faculty. This course provides a broad introduction to machine learning, datamining, and statistical pattern recognition. Topics include: (i) Supervised learning (parametric/non-parametric ...


16

This is a very interesting question. I believe it is getting a lot of up-votes from people who have wondered the same thing and don't know where to begin, whereas you have at least laid out a reasonable-sounding plan. I commend you for that. However, it is not clear to me what you're trying to learn by posting this question. In my opinion, the plan you ...


14

Recently I attended a presentation by the first author of the following paper who gave us quite a creative and illuminating (kind of meta-)use of random forests in Quant Finance: All that Glitters Is Not Gold: Comparing Backtest and Out-of-Sample Performance on a Large Cohort of Trading Algorithms (March 2016) by Thomas Wiecki, Andrew Campbell, Justin Lent, ...


13

In a very, very general sense, what Renaissance Technologies does well [and others try to do, many do less well] is understand where the "true" signal is (i.e. where prices should be) and what is noise (i.e. over-/under-reactions by others in the market) in the total signal of market prices. Generally, trading profits are made by taking the opposing ...


13

As the others have already mentioned, this is a very broad question. Anyway, as a starting point there are some blogs that come to my mind that have some up to date high quality content on these issues from time to time: http://quantivity.wordpress.com/ http://epchan.blogspot.com/ http://www.automated-trading-system.com/ http://intelligenttradingtech....


12

You should consider an unsupervised learning algorithm such as K-nearest neighbor ('KNN'). KNN will measure the distance amongst the observations in your space. You can and probably should consider alternative distance functions (besides euclidean) particularly if you are clustering on features such as returns which have outliers. There are quite a few ...


11

I think R's CRAN Task Views on Machine Learning is an excellent resource for beginners moving to advanced algorithm traders. It is well-structured, broad, up-to-date, and ready-to-use! http://cran.r-project.org/web/views/MachineLearning.html I believe all advanced quantitative traders already know this. But I haven't seen anyone post it here and Flake's ...


11

Here's my favorite example of an intraday strategy on S&P500 futures that at least used to work: Intraday Share Price Volatility and Leveraged ETF Rebalancing I pull it out whenever people start talking about market efficiency. The strategy is very simple: if S&P500 futures are up or down more than 2% on the day with two hours left until close, ...


9

I was just like you when I started out: I had learned a lot about machine learning (mainly neural networks and genetic algorithms/programming) and used it heavily. I also had learned about classic statistics but not nearly as much as about ML. The problem with ML is - as I see it today - that you are often taking a sledgehammer to crack a nut, meaning: ...


9

I would say that most ML methods risk overfitting and it depends very much on the asset class. The only area where more sophisticated ML methods such as deep learning appear to make a major difference is in cash equities, where the feature space is very rich (NLP, news and announcements, corporate earnings, other financials) and the data is relatively good, ...


8

I have been learning more about speech recognition motivated by its application to financial forecasting. I have identified a couple connect points. Turns out each of these tools can and are regularly used in financial modeling as well. Use of Markov Models Use of Fourier transforms (sine/cosine decompositions) Use of component analysis


8

First, we are few quants and academics to use the full toolkit of machine learning: stochastic algorithms, to optimal trading. Here are at least two papers: Optimal split of orders across liquidity pools: a stochastic algorithm approach, Sophie Laruelle (PMA), Charles-Albert Lehalle, Gilles Pagès (PMA) Optimal posting distance of limit orders: a stochastic ...


8

One excellent resource is to try Kaggle and to examine some of the competitions, some of which are specifically on the application of machine learning to credit scoring. https://www.kaggle.com/c/GiveMeSomeCredit You wil see that the winning solution is made public, including source code and output. https://github.com/IdoZehori/Credit-Score/blob/master/...


7

From this site's perspective, I think nothing would be better than a ML.SE. Finally, we got one awhile ago. UPDATE: Unfortunately, Machine Learning is merging into Cross Validated. To learn more detail, click here." I have no idea why SE admin was rush to merge ML into CrossValidated. Not a fan of it (Orz). I personally prefer a separate site. FYI, http:...


7

One relevant paper is: Shenoy, C. and Shenoy, P.P., Bayesian network models of portfolio risk and return, 1999. PDF


7

It's probably because of the strong long-standing statistical underpinnings in economics and econometrics, and overall, risk prediction. For example, look at current research with fat-tail distributions and calculations for Expected Tail Loss (ETL), etc. These studies fit Student's t, Normal, Stable, and Pareto probability distributions to data and report ...


7

There is at least one clear area of application of ML in Q quant finance, it is the LSM algorithm invented by Longstaff, Schwartz and Carriere in the late 1990s for the valuation of callable exotics in the context of Monte-Carlo simulations, and widely adopted for more recent bank-wide risk calculations like CVA. In order to estimate the continuation value ...


6

People seem to think that using ML is going to circumvent the process of actually learning to trade, it doesn't. ML can be used to refine trading ideas, but it doesn't generate them, you need to use your brain for that.


6

In 2010 Informs and Kaggle organized time series prediction contest. The methods used by competitors are described here.


6

I'm currently working on this task, to apply machine learning to stock trading. However, the concerns raised in other answers are major obstacles. So, I'm taking a different tact. My strategy is more akin to teaching a car to drive - the machine learning is not based on the underlying data, but rather on the driver's reaction to the data. So based on what ...


6

do input attributes need to be scaled? No. It is not required. It is only a heuristic [1]. It is primarily motivated because of the following: From the Feature Scaling article: Since the range of values of raw data varies widely, in some machine learning algorithms, objective functions will not work properly without normalization. For example, the ...


6

Such a complex question... Geometric Brownian Motion (GBM) will not typically work to aid one finding strategies based on technicals, as the pursuit of the technical trader is to find market deviations from a random walk. However, some strategies, for example a "take profit/stop loss" strategy can work, (or at a minimum one can change the risk/reward ...


5

Sorry, but despite being used as a popular example in machine learning, no one has ever achieved a stock market prediction. It does not work for several reasons (check random walk by Fama and quite a bit of others, rational decision making fallacy, wrong assumptions ...), but the most compelling one is that if it would work, someone would be able to become ...


5

One possibility worth exploring is to use the support vector machine learning tool on the Metatrader 5 platform. Firstly, if you're not familiar with it, Metatrader 5 is a platform developed for users to implement algorithmic trading in forex and CFD markets (I'm not sure if the platform can be extended to stocks and other markets). It is typically used for ...


5

The Brown et al. paper and its connection with trading is discussed here: http://jochenleidner.posterous.com:80/from-speech-recognition-to-trading (mirror)


5

You need to define the parameters over which you are searching (i.e. # of bands, slope of trendline, some function relating slopes to trendline, etc.). Then you can use your favorite optimizer to identify which parameters satisfy your P&L objective. Of course, your approach is a surefire way to lose money since this curve-fitted model will not ...


5

Speech recognition signal processing is complex and possibly similar to the complexity of financial markets. They are similar as per characterictics the non stationarity, noise types and other aspects such us the existence of a cepstrum etc conceptual frequency and the grammar to construct and articulate concepts is not evenly and randomly distributed; so ...


Only top voted, non community-wiki answers of a minimum length are eligible