Quantitative Finance Stack Exchange is a question and answer site for finance professionals and academics. It only takes a minute to sign up.
Sign up to join this community
Are there any methods of prediction (machine learning, regression, etc.) which are designed to maximize the rank correlation (spearman correlation, kendall's tau, etc.) of your prediction with your independent variable?
I am trying to predict a continuous numerical variable with a given set of inputs, and I am more concerned that the rank-correlation of predictions is high than the pearson correlation.
The simplest way is to transform your dependent variable from returns into rank-space and then use ordinary least squares regression.
A more complex technique would involve setting up an optimization problem where you maximize the spearman correlation between your vector of predictions and actuals. More explicitly, the objective function in the optimizer is the spearman rank correlation function. The optimizer will search over the set of parameters to some model specification (for example, if you use a linear factor model it will identify a set of betas) that maximize the spearman correlation. Of course, there are many potential model specifications so this approach is a bit open ended.
Another simple approach would be to use a cumulative gains and lift chart to select amongst competing regression models.
Or you can use a discrete dependent variable to attempt to predict which quantiles an instrument is likely to fall within. A simple example would be to develop a logistic regression model where you predict 1 or 0 (let's say 1 means Up, and 0 means down). Then you can use the probability outputs generated when you score data using your logistic model to rank-order your predictions (vs. using regression outputs which may be more sensitive to outliers).
