1
$\begingroup$

As a beginner, I'm learning how to make good trading strategies. One of the things to consider for reliable backtesting is the Minimum Backtest Length, whose selection is basically a tradeoff:

Too short a backtest duration $\implies$ statistically unreliable backtest

Too long a backtest duration $\implies$ underlying dynamics of the market may have changed.

I don't have a concrete idea of what "underlying dynamics of the market" means in the context of trading strategies. Secondly, from the first answer to this question: Backtesting Period, I gathered that the "market dynamics" are different for trading strategies of different frequencies. So the underlying dynamics relevant to an HFT strategy change more frequently than those relevant to a strategy with weekly turnover.

Firstly, what exactly is meant/covered by market dynamics that affect a particular trading strategy?

More importantly, could someone please explain exactly how one can identify when and why the underlying dynamics of the market changed?

$\endgroup$
1
$\begingroup$

Markets are adaptive, highly dynamic systems that depend on the input of a multitude of risk attitudes, investment ideas, trading frequencies, economic expectations and much more.

One way of understanding market dynamics is through assuming you have a parameterized model for explaining or trading the market, $m$, that depends on a set of parameters $\eta_i$, $i \in \{1,2,\ldots, n\}$. Such a model could be the CAPM model where the interest rate of an asset $r$ is decided via $$r = \alpha + \beta r_{market} + \epsilon, \quad\epsilon \sim N(0,\sigma)$$ where your set of parameters is $\{\alpha, \beta, \sigma\}$. If you go on to estimate this model (via OLS, for example), you have an implicit assumption that your parameter set won't change significantly in the out-of-sample period, otherwise the model is not usable for trading as the world you have looked at is very different than the world in the future, and it is only good for explaining the past. If you are using daily return data and you estimate the $\beta$ variable for the last 30 years, chances are you will have gone through a number of different market-types that all are lumped together in your static parameter estimate. Thus you often want to create models that change dynamically over time, such as using rolling estimation of $\beta$. And of course, the above model is pretty much useless at extremely high frequencies, and you will not care too much about the interest rate policy when you are working with tick data.

It also boils down to what a trading strategy is supposed to $do$. If we assume you aren't market making, then you will be exploiting some regularly occurring pattern or anomaly via some sort of indicator that gives you signals. This pattern can disappear if other traders latch onto it, or if the market dynamics change due to an external event.

If you want to detect regime shifts in the parameters, there are a number of methods. This paper gives a very large number of different methods developed for this purpose. Some of the most popular in finance are (Hidden) Markov Models and changepoint analysis. You can find good questions on this under the market-regime tag here.

I will note that I don't believe that a backtest can be too long if you are at lower frequencies, as the longer data will just give you more confidence that your model actually could be useful in a multitude of market regimes rather than just the recent past. Many people go to great lengths to generate artificial data with particular properties to vastly increase their backtest length, synthetically.

$\endgroup$
  • $\begingroup$ Thanks, that's a very nice explanation! So from what I understand, the market dynamics could be expressed in the form of a model, for example CAPM. A shift or change in the "market dynamics" could then be detected by estimating the model parameters at different times and noticing at what point a significant change in those parameters occured. My follow-up questions is, how do we know which model to use? Based on what factors do we select our model? $\endgroup$ – u23 Feb 12 '17 at 21:09
  • 2
    $\begingroup$ Yes, that is correct - it is useful to think in terms of models, but of course the real world is way too complicated for us to model everything. A simple paper on the $\beta$ estimation problem is nera.com/content/dam/nera/publications/2016/…, where the author contrasts two different methods of estimation for data with regime shifts. Knowing what model to use, how to estimate the parameters, and knowing which inputs are the best for predicting or explaining some part of the market - that is where the traders' skill and creativity comes in. $\endgroup$ – Forgottenscience Feb 12 '17 at 21:20
  • $\begingroup$ Suppose that I use a strategy of assigning weights to stocks on the basis of their returns. I could use the CAPM model to model the market dynamics, but I'm sure there are other, lesser known models that give an expression for stock returns, but with different inputs. Would a decision on which of those models to use (for representing market dynamics) also depend on the trader's experience and skill? $\endgroup$ – u23 Feb 12 '17 at 21:59
  • $\begingroup$ One last thing - I'm an absolute beginner to all this. So do you know of any resources from where I can learn about what models or inputs are used for what strategies? In other words, how do I build up that "trader's skill" of identifying which model uses the same inputs as the ones that affect a trading strategy? I guess I could refer research papers, but there are so many of them that it's overwhelming to find the "right" research paper/article. $\endgroup$ – u23 Feb 12 '17 at 22:01
  • 1
    $\begingroup$ First comment: In a sense, yes. Knowledge of the mathematical modelling possibilities for each problem is useful domain knowledge. Second comment: I would read Robert Carver's Systematic Trading for a very good overview of general systematic trading. For more specific information, maybe read Ernest Chan's Algorithmic Trading, which covers basic momentum and mean reversion ideas. Essentially, you need to figure out what anomaly you want to exploit - and then subsequently figure out how to extract the signal you want to trade. Finally, google Marco Dion's interview with Active Trader. $\endgroup$ – Forgottenscience Feb 12 '17 at 22:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.