2
$\begingroup$

Trading strategies often have many degrees of freedom. As a toy example let's say you have two moving averages (MA) which trigger a trade each time they cross each other: There are at least two parameters which could be optimized, namely the length of each MA.

Now, you would not want to bet your money on a strategy that just by chance found that you have to take 57 days for one and 243 days for the other MA (see data snooping bias). What you want to see is that the strategy as such is sound per se, i.e. robust und not dependant on the exact parameter settings.

One way to go for two parameters is to plot a heatmap with the two parameters as axes and a colour coding for the respective return of the strategy in a backtest. If you only see noise and many convoluted regions of different return levels this is a good sign that this is not a robust strategy. If there are bigger regions of positive returns these regions merit further investigation.

My question
What are established methods to find robust regions of multidimensional parameter combinations in trading strategies? The challenge here: You obviously cannot visualize more than three degrees of freedom and you have to make these ideas mathematically rigorous.

Multivariate kernel density estimation comes to mind but this is just my first idea. I am thankful for every lead, reference and code example (preferably in R).

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ I agree with @madilyn. IMO, you should be more scared about the generalisation capability of your model than its robustness. I would therefore recommend cross-validation techniques since it is well know that the error measured on a backtest is an optimistically biased estimator of the expected generalisation error of your model (since your model is generally chosen so as to minimise the former error). $\endgroup$
    – Quantuple
    Commented Mar 8, 2017 at 9:05
  • $\begingroup$ @Quantuple: I of course know that, yet the above could be a first step to understand more about the behaviour of the strategy within its parameter space. If the strategy is not even robust you need not worry about generalizability. $\endgroup$
    – vonjd
    Commented Mar 8, 2017 at 9:09
  • $\begingroup$ @Quantuple: Another thing is of course that using cross-validation on ordered data like time series has challenges of its own - see also here: stats.stackexchange.com/questions/14099/… $\endgroup$
    – vonjd
    Commented Mar 8, 2017 at 9:20
  • 1
    $\begingroup$ What I'm saying is on the contrary that if a strategy does not generalise well (which is eventually the only thing you are interested in) you should not even care whether the numerical estimation procedure of its parameters is robust. In other words, by picking a "robust" solution (here focusing on local extrema of the objective function whose neighbourhood in the parameter space is flat instead of a global extrema), you effectively restrict your model class, which can potentially lead to restricting yourself to strategies that are bound to be poor at generalising. $\endgroup$
    – Quantuple
    Commented Mar 8, 2017 at 9:52
  • 2
    $\begingroup$ ... and of course you are right, CV for time series data is challenging, at least the standard $K$-fold CV. So I guess that what I'm saying is that what you suggest is OK but should be part of the "strategy identification" and the whole should be run through (cross-)validation to have a good idea of the generalisation error. $\endgroup$
    – Quantuple
    Commented Mar 8, 2017 at 9:59

1 Answer 1

Reset to default
1
$\begingroup$

Not exactly the answer you're looking for: It's not obvious that a region of stability is a desirable property.

One can trivially construct an example where this is true: suppose the actual generation function of your target is $f: x_t \mapsto 2x_t+1$ over $\mathbb{R}$, and you have a signal $s$ of one parameter $s\left(p,x_t\right)=\left(p^{e} \mod 3\right) \cdot \left(2x_t+1\right)$ where $s$ is defined over the domain of $\mathbb{R}^{+} \times \mathbb{R}$. If you try to learn some $\hat{f}\left(p,\cdot \right)=s\left(p,\cdot \right)\approx f(\cdot)$ by brute force grid search of $p$ that minimizes the RMSE followed by some smoothing transformation of your grid, you will easily end up with a model that doesn't generalize well, even though a naive optimizer might converge to $p=1$ easily.

So whatever fanciful stability function that you choose, I would test your hypothesis first.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.