9

I would say that most ML methods risk overfitting and it depends very much on the asset class. The only area where more sophisticated ML methods such as deep learning appear to make a major difference is in cash equities, where the feature space is very rich (NLP, news and announcements, corporate earnings, other financials) and the data is relatively good, ...


7

There is at least one clear area of application of ML in Q quant finance, it is the LSM algorithm invented by Longstaff, Schwartz and Carriere in the late 1990s for the valuation of callable exotics in the context of Monte-Carlo simulations, and widely adopted for more recent bank-wide risk calculations like CVA. In order to estimate the continuation value ...


5

The code below is written in Wolfram Mathematica. For example, we have some training data. And we are trying to predict: long (1) or short (0). SeedRandom[0]; n = 10000; X = RandomReal[{-1, 1}, {n, 100, 5}]; Y = RandomInteger[{0, 1}, n]; net = NetChain[ { LongShortTermMemoryLayer[64], SequenceLastLayer[], ElementwiseLayer[Ramp], LinearLayer[...


4

There are a few exclusions that I have commonly seen: Excluding thinly traded stocks. The price that shows up in your data feed may not relate to actual tradable prices. Filtering for ADR/Pink locals. You can find stocks listed in multiple places in ways that would lead you to think that they are great for pairs trades when actually they are the same ...


4

This question is broad, and the normalisation strategy is going to depend on the nature of your indicator. Assuming the technical indicators are a time series, then two simple approaches for normalising your data are: Calculate the difference between each time step. If you are feeding multiple signals into a neural network you should confirm the values ...


3

Here is one recipe, in case you can live with Spearman rank correlation. (Which you should: linear correlation is often not appropriate in the non-normal case. And in the normal case, there is almost no difference between the two correlation types.) Generate samples of your $k$ features with all the desired attributes. These samples may be random or ...


3

You should consider the stages of the default process instead of a binary "default", where there are various points the borrower is able to cure the loan. In a traditional credit model, the general process is to predict the state of the loan and then predict transitions between stages over the life of the loan. This is done by simulating macro variables (...


3

Z. Jiang, D. Xu, J. Liang, in A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem. demonstrate a Deep RL framework for Trading. The approach is based on Tensor flow and uses the ideas similar to the Open AI Gym used by Deepmind for video games! In my blog Optimizing a Portfolio of Cryptocurrencies with Deep Reinforcement ...


2

You can use rank correlation in lieu of Pearson correlation to remove that linearity basis. And if tail dependence is of particular interest, one way to look at it is using a t-copula and check the degrees of freedom.


2

You should probably look into Poon, Rockinger and Tawn (2003). In particular check how they build the $\chi$ and $\bar{\chi}$ measures of correlation which account for extreme events in up or down markets. From their paper: "The conventional dependence measure, the Pearson correlation $\rho$, is constructed as an average of deviations from the mean. It ...


2

If you truly don't need the time of the earnings report, you can use Tradier. https://developer.tradier.com/documentation/markets/fundamentals/get-calendars


2

Cholesky (or SVD or any other approach based on matrix multiplication) only works for normal distributions, which your features cannot be, given that they have values within finite intervals. To see why Cholesky does not work, assume two additional features, which are independent uniform $(U_1,U_2)$. Now you want to create features with correlation $\rho$ ...


2

Whilst reading this I realized that it would be a really good application for meta-labeling. The idea behind meta-labeling is to build a secondary model that determines if the signals {0, 1} from the primary model are correct or not. By doing this the secondary model outputs a value between 0 and 1 indicating how confident the model is that the primary ...


1

The following post on cross-validated has quite a good answer: "Random forest uses bagging (picking a sample of observations rather than all of them) and random subspace method (picking a sample of features rather than all of them, in other words - attribute bagging) to grow a tree. If the number of observations is large, but the number of trees is too ...


1

Given you have a database that stores this data daily, you could write a short python script to apply your screening and email you the daily rankings or scores. I think you can even do this in Excell if you have a Bloomberg or TR terminal. I am pretty sure that you can. If you want to backtest the performance of such a strategy then I think using ...


1

Since the correlation matrix $C=LL^{\top}$ is also $C=U^{\top}U$, where $U$ is the upper triangular matrix, rather than $L$ the lower triangular matrix, you can transform an uncorrelated features matrix $F$ containing features 1, 2 and 3 in its columns by multiplying this $F$ matrix with $U$, giving the correlated features matrix $F_c$: $$F_c = F U $$ In ...


1

A good beggining could be the paper of Gu, Kelly and Xiu (2018).


1

Data pre-processing is often a very important (if not the most important) step in a machine learning algorithm. Decisions trees are often an exception that they can work well without any pre-processing. But they may work better if you can identify some processes that might improve the quality of the decision detection. As an example of other machine ...


1

Merton model has been highly criticized in academic literature for its accuracy, though it provides good ranking of credit risk, it fails to quantify it. I'd say use machine learning or better yet deep learning. I used a recurrent neural network with time series inputs like amount due and changing monthly income among many more. It provided a good estimate ...


1

It can , it depends on your trading strategy. Let's say model1 predicts relative return as 0.1 for next day and model2 predicts relative return as 0.3, while actual return is 0.15. Model1 has lower RMSE error. If your trading strategy is to buy when model predicts positive next day return,and do so in volume proportional to model prediction, then you ...


1

My answer to a similar question on the Cross Validated forum link here might be useful. In a nutshell you need to optimise for profit and not MSE - the two are not necessarily one and the same.


1

There are many ways to go about this, for example: You can build one global model using combined dataset (less frequent data will essentially carry the precious observed value until the next observation). Problem that you would face with this approach is the one of the data sources may be less predictive so you might not get a balanced model. Build two ...


1

Disclaimer: these are just opinions, I do not necessarily have authoritative knowledge in this topic. If you consider the traditional Sharpe definition: $$S = \frac{reward}{risk}$$ where reward is the expected return (above risk free rate) and risk is the standard deviation of reward, it is not clear to me how your augmented Sharpe ratio is related to this....


1

A completely different statistical approach is to pose your own machine learning problem: 1) Collect a set of full data where you have data values available for all instruments on any given day. 2) Propose a machine learning model that will devise its own optimised parameters for the task of regressing any missing data. 3) From your set of good data ...


1

For a VaR calc you might not want to interpolate missing values. By doing that you are inherently editing the returns distribution; potentially this will make a VaR look better or worse. Not good if your goal is an accurate risk distribution. Its worth considering what a missing value signifies. There are two cases. A missing value or unchanged value can ...


1

If you consider hedging a data mining problem you might arguably construct the following scenario: Given a portfolio of positions, find the parameters $\beta_i$ representing the weights of new (hedging) positions from a set of instruments $\{I_i\}$, such that the variance of the PnL of the combined portfolio is minimal. If you considered a rather trivial ...


1

I think the bible of machine learning in finance has become: Advances in Financial Machine Learning by Marcos Lopez de Prado 2018.


1

The correlation matrix is a very important part of modeling stock returns. It is often better to build a model that takes in multiple assets features so that it can use this correlation to its advantage. A good example of this is a VAR model from econometric. A great example in the machine learning context is the paper titled Empirical Asset Pricing via ...


1

From what I have read, there are 3 popular algorithms for financial time series. Random Forests and SVMs, then followed by Neural Network Architectures. There are a couple of good papers, to name a few: Empirical Asset Pricing via Machine Learning Deep neural networks, gradient-boosted trees, random forests: Statistical arbitrage on the S&P 500 An ...


Only top voted, non community-wiki answers of a minimum length are eligible