The Stack Overflow podcast is back! Listen to an interview with our new CEO.
9

I would say that most ML methods risk overfitting and it depends very much on the asset class. The only area where more sophisticated ML methods such as deep learning appear to make a major difference is in cash equities, where the feature space is very rich (NLP, news and announcements, corporate earnings, other financials) and the data is relatively good, ...


7

There is at least one clear area of application of ML in Q quant finance, it is the LSM algorithm invented by Longstaff, Schwartz and Carriere in the late 1990s for the valuation of callable exotics in the context of Monte-Carlo simulations, and widely adopted for more recent bank-wide risk calculations like CVA. In order to estimate the continuation value ...


5

The code below is written in Wolfram Mathematica. For example, we have some training data. And we are trying to predict: long (1) or short (0). SeedRandom[0]; n = 10000; X = RandomReal[{-1, 1}, {n, 100, 5}]; Y = RandomInteger[{0, 1}, n]; net = NetChain[ { LongShortTermMemoryLayer[64], SequenceLastLayer[], ElementwiseLayer[Ramp], LinearLayer[...


4

There are a few exclusions that I have commonly seen: Excluding thinly traded stocks. The price that shows up in your data feed may not relate to actual tradable prices. Filtering for ADR/Pink locals. You can find stocks listed in multiple places in ways that would lead you to think that they are great for pairs trades when actually they are the same ...


4

This question is broad, and the normalisation strategy is going to depend on the nature of your indicator. Assuming the technical indicators are a time series, then two simple approaches for normalising your data are: Calculate the difference between each time step. If you are feeding multiple signals into a neural network you should confirm the values ...


3

Here is one recipe, in case you can live with Spearman rank correlation. (Which you should: linear correlation is often not appropriate in the non-normal case. And in the normal case, there is almost no difference between the two correlation types.) Generate samples of your $k$ features with all the desired attributes. These samples may be random or ...


3

Z. Jiang, D. Xu, J. Liang, in A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem. demonstrate a Deep RL framework for Trading. The approach is based on Tensor flow and uses the ideas similar to the Open AI Gym used by Deepmind for video games! In my blog Optimizing a Portfolio of Cryptocurrencies with Deep Reinforcement ...


3

You should consider the stages of the default process instead of a binary "default", where there are various points the borrower is able to cure the loan. In a traditional credit model, the general process is to predict the state of the loan and then predict transitions between stages over the life of the loan. This is done by simulating macro variables (...


2

Whilst reading this I realized that it would be a really good application for meta-labeling. The idea behind meta-labeling is to build a secondary model that determines if the signals {0, 1} from the primary model are correct or not. By doing this the secondary model outputs a value between 0 and 1 indicating how confident the model is that the primary ...


2

The standard approach for cross-validation is to work with three datasets the training set, the validation set, the test set. You train on the training set (no surprise), and you use the validation set to decide for your hyper-parameters. Then you retrain, test on the test set (no surprise again), and it is done. My advice here is to consider that the ...


2

One option could be to use an early stopping mechanism, e.g. stop when the average reward per episode on a validation set stops increasing. In practice, I found this to be a bad idea as you generally don't have enough data to be able to afford a validation set when using daily data (in addition to the test set). If you are dealing with higher frequency data ...


2

Given you have a database that stores this data daily, you could write a short python script to apply your screening and email you the daily rankings or scores. I think you can even do this in Excell if you have a Bloomberg or TR terminal. I am pretty sure that you can. If you want to backtest the performance of such a strategy then I think using ...


2

Cholesky (or SVD or any other approach based on matrix multiplication) only works for normal distributions, which your features cannot be, given that they have values within finite intervals. To see why Cholesky does not work, assume two additional features, which are independent uniform $(U_1,U_2)$. Now you want to create features with correlation $\rho$ ...


1

The problem you describe can be handled as an optimization problem: evolve a program such that it maximizes some performance measure. The technique you may want to look into is called "Genetic Programming". For a financial application see for example Single versus Multiple Tree Genetic Programming for Dynamic Decision Making.


1

The following post on cross-validated has quite a good answer: "Random forest uses bagging (picking a sample of observations rather than all of them) and random subspace method (picking a sample of features rather than all of them, in other words - attribute bagging) to grow a tree. If the number of observations is large, but the number of trees is too ...


1

Since the correlation matrix $C=LL^{\top}$ is also $C=U^{\top}U$, where $U$ is the upper triangular matrix, rather than $L$ the lower triangular matrix, you can transform an uncorrelated features matrix $F$ containing features 1, 2 and 3 in its columns by multiplying this $F$ matrix with $U$, giving the correlated features matrix $F_c$: $$F_c = F U $$ In ...


1

With ML, you're looking to identify patterns in your inputs that result in your output(s). Thus you collect all the outputs you are hoping to be able to identify later, and the inputs which correspond to those outputs (i.e just before the output was generated), collect as many as possible of these relationships, stick them into a ML model and hope you've ...


1

Data pre-processing is often a very important (if not the most important) step in a machine learning algorithm. Decisions trees are often an exception that they can work well without any pre-processing. But they may work better if you can identify some processes that might improve the quality of the decision detection. As an example of other machine ...


1

Merton model has been highly criticized in academic literature for its accuracy, though it provides good ranking of credit risk, it fails to quantify it. I'd say use machine learning or better yet deep learning. I used a recurrent neural network with time series inputs like amount due and changing monthly income among many more. It provided a good estimate ...


1

It can , it depends on your trading strategy. Let's say model1 predicts relative return as 0.1 for next day and model2 predicts relative return as 0.3, while actual return is 0.15. Model1 has lower RMSE error. If your trading strategy is to buy when model predicts positive next day return,and do so in volume proportional to model prediction, then you ...


1

My answer to a similar question on the Cross Validated forum link here might be useful. In a nutshell you need to optimise for profit and not MSE - the two are not necessarily one and the same.


1

There are many ways to go about this, for example: You can build one global model using combined dataset (less frequent data will essentially carry the precious observed value until the next observation). Problem that you would face with this approach is the one of the data sources may be less predictive so you might not get a balanced model. Build two ...


1

Disclaimer: these are just opinions, I do not necessarily have authoritative knowledge in this topic. If you consider the traditional Sharpe definition: $$S = \frac{reward}{risk}$$ where reward is the expected return (above risk free rate) and risk is the standard deviation of reward, it is not clear to me how your augmented Sharpe ratio is related to this....


1

A good beggining could be the paper of Gu, Kelly and Xiu (2018).


1

A completely different statistical approach is to pose your own machine learning problem: 1) Collect a set of full data where you have data values available for all instruments on any given day. 2) Propose a machine learning model that will devise its own optimised parameters for the task of regressing any missing data. 3) From your set of good data ...


1

For a VaR calc you might not want to interpolate missing values. By doing that you are inherently editing the returns distribution; potentially this will make a VaR look better or worse. Not good if your goal is an accurate risk distribution. Its worth considering what a missing value signifies. There are two cases. A missing value or unchanged value can ...


1

I think the bible of machine learning in finance has become: Advances in Financial Machine Learning by Marcos Lopez de Prado 2018.


1

The correlation matrix is a very important part of modeling stock returns. It is often better to build a model that takes in multiple assets features so that it can use this correlation to its advantage. A good example of this is a VAR model from econometric. A great example in the machine learning context is the paper titled Empirical Asset Pricing via ...


1

From what I have read, there are 3 popular algorithms for financial time series. Random Forests and SVMs, then followed by Neural Network Architectures. There are a couple of good papers, to name a few: Empirical Asset Pricing via Machine Learning Deep neural networks, gradient-boosted trees, random forests: Statistical arbitrage on the S&P 500 An ...


1

OK, in for a penny, in for a pound :-) First, let us assume that you have symmetrical critical levels higher and lower (call them “target” and “stop” if you will). Only in this case is the hit rate relevant. Even, then the hit rate is a function of time. If you take a 5d/1w view, then being 51% right is very different to being to being 51% right on a 21d/...


Only top voted, non community-wiki answers of a minimum length are eligible