0
$\begingroup$

Suppose you have created a new trading algorithm:

  • by varying the params of the algorithm, you get a large number of similar trading strategies (e.g. slightly different trigger thresholds, stop loss thresholds, etc.)

  • for each strategy you can calculate the historical daily returns and various historical performance metrics

  • now, you must pick 1 single strategy that will be traded in the market tomorrow (i.e. out-of-sample)

What performance metric will help you select the optimal strategy?

I did some simulations (details below) and for example:

  • I found that ${historical \space average \space returns}$ are a terrible predictor of future performance

  • while ${sharpe}=\frac {daily \space return \space mean}{daily \space return \space dev}$ is the best predictor I found so far

However, I have no idea why it is so and if there are even better predictors. Help!

enter image description here

Improve this question
$\endgroup$
1
$\begingroup$

The sortino ratio is also important for evaluating trading strategies. also the omega ratio.

The question is a poor one though. Each of the mentioned ratios will be the most predictive at predicting ... THEMSELVES! respectively. you don't use the calmar to predict the sharpe. ok, what you are probably actually asking is which of the performance metrics suffers the lowest estimation error. that takes time to understand what exactly each is calculating by breaking up the ratios into their components while going through the hundreds of existing studies that already report their reliability during estimation/prediction. the learning speed of each is of no interest either since none of the metrics listed are computationally intensive. but even after all this: why would any of them give you an edge, in isolation or in combination, in the first place? They are merely post-evaluation tools intended for analyzing realized performance, but unfortunately sold off by fund managers as decision criterions. Two very different purposes.

Due to the estimation error each possesses, and their mis-useage for prediction rather than post-evaluation that they are intended for, it should be clear why none of the performance metrics, or even any sort of combination of them, will ever achieve the ideal "equity curve".

Then bringing in the idea to adaptively change the trading strategy's optimality criterion (sharpe on day 1, calmar on day 2 and sortino on day 3) requires a step in between: a criteria in the middle that identifies the conditions under which a certain one of the performance metrics is selected. That step has not been considered in the write-up so far. Not that it will get you anywhere, but it will turn the whole point of the strategy on its head because the strategy is basically one that does not stick to a fixed game plan but executes based on a much longer decision process that is probably aimless in theory. On day 1 you want the best risk-adjusted performance, and then day 2 you want to control for drawdown? How would that sound for an investment policy? Investors usually just want one thing: high cumulative returns, the more money the better, without exception. After all, thinking that the selection of a certain performance metric for one specific trading interval will consistently make you richer? think again.

The biggest mistake in designing trading strategies is to believe that having a whole toolkit of well-known magical formulas and throwing them under the bus will make a bunch of profit. That's not what finance is about. Step back and draw up a theory as to why what you are doing will accomplish a certain result (consistently, and for a variety of assets that have different statistical characteristics) rather than just plugging and playing whatever you see in a finance textbook.

with all this said,what is the formula there for that "out of sample equity curve"? also, the ideal one doesn't look curved.

Improve this answer
$\endgroup$
7
  • $\begingroup$ @deveralist thanks! the ideal equity curve is straight because I summed up the daily returns, I did not compound them. I don't have the formula of the other 3 OOS equity curves, these are numerical results. You wrote "Investors just want high cumulative returns", this was indeed the goal of my simulation! The chart suggests that selecting the strategy with max historical Sharpe leads to max cumulative returns, on average. I appreciate I assume "past predicts future" in my reasoning, but then how can one design strategies fully blind of the past? Please suggest me a book to do things right $\endgroup$ – elemolotiv May 27 '20 at 7:49
  • 2
    $\begingroup$ ok this is the first i heard here that the objective is to maximize cumulative returns. obviously then the sharpe ratio is not appropriate because it maximizes risk-adjusted returns. the calmar isn't appropriate either because it divides returns by drawdown. historical average returns won't do either because average returns are not cumulative returns. This is what I mean by drawing out your theory carefully. All of the metrics proposed have instantly been ruled out. The fact that the sharpe comes closest to the ideal "curve" is meaningless since ur still not achieving perfect prediction. $\endgroup$ – develarist May 27 '20 at 11:40
  • 2
    $\begingroup$ here's the spoiler: perfect prediction is impossible. everyone knows that past predicts future was never a theory in the first place, it has been refuted for decades in finance, so why pursue it here? if you want to maximize cumulative returns, you set that as the objective function, but even that won't work because the for half a century it has been proven that the mean of asset returns is more difficult to predict than variance (high estimation error) $\endgroup$ – develarist May 27 '20 at 11:45
  • 1
    $\begingroup$ with all of the proposed performance metrics ruled out, because they do not maximize cumulative returns by design, there obviously is no reasoning to be adaptively switching between any of them on a day-to-day basis when each of their objectives do not even reflect the goal here of maximizing cumulative returns $\endgroup$ – develarist May 27 '20 at 11:52
  • 2
    $\begingroup$ based on theory, all reasoning would support going with the performance metric that captures your desired objective function (maximize cumulative returns by maximizing cumulative returns, not risk-adjusted returns). Whether or not this accomplishes the task out-of-sample is where you'll find theory doesn't always measure up, but that doesn't mean you then use a different performance metric that does not reflect the goal objective. do not horse race a bunch of measures that do not even attend to the desired objective in the first place. you will run into artefacts much longer down the road $\endgroup$ – develarist May 27 '20 at 12:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.