Want to select a metric or metrics to compare returns of different investment strategies, for quantitative backtesting, strategy selection, and forward measurement.
Been reading different approaches but not finding anything that seems to be simple, practical, time-normalized, and provide statistical projections so could use some help.
Common Approaches
MEAN
Mean Return seems to be by far the most commonly used and reported approach: just measure the mean return over some period of time or several periods of time, and either report it, or compare it with a benchmark. This provides no indication of the statistical likelihood of that mean return being reproducible in the future because there is no sampling. There is also no correction for risk.
RATIOS
Sharpe Ratio and similar. There are several approaches that focus on the mean return of an investment vs a benchmark divided by a measure of variance, such as:
Sharpe Ratio which divides/normalizes mean by standard deviation: https://www.investopedia.com/terms/s/sharperatio.asp
Sorontino Ratio which divides/normalizes mean by "downside deviation" https://www.investopedia.com/terms/s/sortinoratio.asp
- Calmar normalizes by a measure of maximum draw down: https://www.optimizedportfolio.com/risk-adjusted-return/
These ratios are attempts to normalize reward by measures of risk. However, they do not provide any kind of statistical estimate of the likelihood of future reproducibility or confidence interval of expected returns based upon sampling of past performance based upon some model. Typically, since these ratios are nonetheless used to compare performance with an eye towards the future, the default unspecified but implied model seems therefore to be either 'and this may be true in the future too', or 'Past Performance Is No Guarantee of Future Results', neither of which seem quantitatively helpful. Also, these ratios are not well normalized for time, since for example the Sharpe Ratio will produce different values depending on the sampling interval of the data used (hourly, daily, monthly, yearly) while the fundamental investment strategies being compared haven't changed. Also, they don't make apparent how to correctly normalize for / account for the relevant investment hold time horizons for the strategies that are being compared, eg performance will be different and maybe more favorable for a very long hold time even for a strategy with a lot of short-term variability.
HYPOTHESIS TESTS
There are also basic statistical testing approaches to compare the likelihood that one investment strategy is different/better than another one. These include simple tests such as t-tests producing p values that estimate the probability that one investment 'is likely to beat another based on past performance', using sampling of past performance and statistical models to predict the future, or more meekly at least to measure the properties of the past sample (eg normality, stationarity):
However, these simple statistical metrics don't indicate the magnitude of the effect in absolute units (eg mean growth rate), only the likelihood of something "recurring"/not being due to chance. They also don't normalize for / account for time invested.
BAYESIAN APPROACHES
There is also the general class of statistical approaches that uses a set of beliefs/priors to estimate likely outcomes: https://www.investopedia.com/articles/financial-theory/09/bayesian-methods-financial-modeling.asp
If there is a standard, widely-used, simple, practical way to use Bayesian methods for comparing investment returns of strategies for testing, which seems like it might be great, I wasn't able to find it.
BOOTSTRAPPING / COMPUTATIONAL STATISTICS
Bootstrap methods can also be used to avoid formal statistical models. For example, it is possible to compare two investment strategies by repeatedly taking samples from the returns of each of the two strategies for short periods of time to form two distributions of returns, one for each strategy, and then quantitatively compare the two distributions, for example to discover how often the average return from one strategy's distribution of returns is larger than the average return from the other strategy's distribution of returns. This has the advantage of being more 'model free', and allows both time horizon and prediction to be taken into consideration, but does not provide an obvious approach to normalization for either one. Example:
https://www.investopedia.com/terms/b/bootstrapping.asp
WHAT IS SOUGHT Looking for a metric (or several) to use to practically, quantitatively compare investment strategies for backtesting and forward prediction that are intended to capture 'which strategy to select based on the likelihood of a "better" risk-adjusted probability of return'. I am aware that this is not a fully/well-specified objective, as there are many potential definitions of "better", which is kind of the point of this question. I want to learn how people who've thought about this longer and come up with better specified and useful quantitative metrics for comparing investment strategies have done it, eg for back testing, and whether there are simple metrics that have meaningful advantages over the most common approaches mentioned above. On a practical level, aware that books have been written on this subject and there is a whole field of knowledge. Looking for simple, practical metrics or approaches for real-world use in testing.
Thank you for your help
