Take the 2-minute tour ×
Quantitative Finance Stack Exchange is a question and answer site for finance professionals and academics. It's 100% free, no registration required.

I'm using around 130 technical indicators for 100 different companies. Each company's stock price moves in a different range, see FTSE 100. In addition, each technical indicator moves in a different range as well, ie some goes between 0-1, other 0-100 and others move with the price. This is the list I'm using http://ta-lib.org/function.html

I'd like to feed this into a machine learning algorithm where I predict the relative price movement of the stock price the next day. I use a logistic loss for profit optimization and two regularizer terms, one between companies, and the other between time periods. This is unimportant for now.

Rather, what I'm asking about is how to normalize the input data? I've tried zscore, rate of change, absolute differences, and various combination of these, but I'm not sure which is the right approach. Also, I assume that first I need to calculate the indicators and then normalize that data. or would these indicators make sense on normalized data?

share|improve this question
4  
I'm not sure there is one right approach. –  John May 7 '13 at 14:34
    
@John what would be a right approach? –  siamii May 7 '13 at 14:36
    
My opinion: PCA over these indicators after having z-scaled observations to make covariance matrix equal to correlation matrix. –  Lisa Ann May 8 '13 at 8:56
    
There will be too much correlation (multicollinearity) between all of your indicators. You could reduce the dimensions using PCA, but that's a linear technique. There may be better payoff using non-linear dimension reduction techniques (diffusion maps, Laplacian eigenmaps, locally preserving projections, Sammon's mapping, self-organizing maps). I have found that among 9 non-linear methods, SOM works the best for classification analysis. However, sounds like you are doing function approximation. –  user5626 Jul 7 '13 at 17:13
    
What I do is put all indicators in bins. Some like volume have to be min(log(vol),some max), otherwise one high value will make bins that are too large. RSI types are easy as you know the max and min beforehand. For the rest you have to choose an appropriate max. –  brian Aug 6 '13 at 14:39

2 Answers 2

Scale and range are your biggest issues. If one input has values which range from e.g. 2300-3500, and another from 0 to 18, then the large scale of the first will swamp the other and provide greater informativity into your learning algorithm. Therefore, normalize into range [0,1] or mean-zero standardize - like you have already done. Be careful with mean-zero standardization, however, since means only apply to skew-zero normal distributions, and not to log-normally distributed (right-tailed) skewed distributions. You could input the ranks of your skewed features values, but ranks are known to be rectangularly distributed, so convert ranks to percentiles, and then to van der Waerden scores (which are standard normal distributed without no skewness).

share|improve this answer

There is no right approach a priori. Try all approaches that make decent sense and pick the one that maximises a utility function on out of sample PnL and risk (or some similar decision rule).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.