# Is it statistically valid to ignore "irrelevant" stock prices during backtesting?

As an amateur trader, I have reasonable success with my current end-of-day trading systems.

All my systems are based on at least ONE year of "in-sample" data for backtesting, followed by 6-months of "out-of-sample" data.

So my question is: Could I take TWO years of "in-sample" data, but ignore any data for periods in which I would not have traded due to the market conditions?

(I used the "data" tag, and searched back to November 2011. I searched all posts for "Historical data".I could not find any related posts.)

Example: Assume I want to trade a "long-only" system, in an "upward-trending" market.

Assume the chart for the previous two years looked like a "W". The first 6 months trends down, then 6 months up, then down, then up.

Can I simply ignore the first and third 6-month periods, and use only the second and fourth 6-month periods?

Reason: If the market was trending down, I would not trade. So, why should my system be based on any previous market data that was trending downwards?

Method:

• In the "W" example above, copy and paste the second 6-month data into Excel.
• Ignore the third 6-month data.
• Copy and paste the fourth 6-month data into Excel.
• But adjust the entire second batch of data upwards so that it joins seamlessly to the end of the first batch. (Possibly by making the first price of the second batch equal to the last price of the first batch.)
• So, the result would be a 1-year data series that only trended upwards.

I would be most grateful for any comments.

It all depends to me on whether your system can categorize a price series as "upward trending."

My first impression was that this is a valid point. So I was about to defend my trend indicator (TE). But, on reflection, I now think it's a distraction. It occurred to me that people such as engineers, IT troubleshooters, and people who conduct human trials always attempt to isolate the "common factor(s)" between two sets of experiments. After that, any differences between the results of the experiments will depend only on differences between the components.

In my case, I think the TE (good or bad) is a common factor. It has already been optimized on a watchlist using 3 years of "undoctored" market data. I use that TE for any system based on that watchlist. (For other watchlists, I have different TEs.)

So the TE will be common to both sets of experiments (doctored data versus undoctored data). For the doctored data, obviously I need to extend the starting point further back in time in order to capture the required number of bars (days) of doctored data.

Here are some statements that I have seen regarding the "selection" or "cherrypicking" of data for backtesting. Unfortunately, I have no links, because I tended to agree with all these statements. (I'm more likely to keep links for statements that I disagree with, for later research.)

Don't assume that a system that works with one of the following will work with the other:

• US stocks versus Australian stocks.
• A portfolio of Financials versus a portfolio of Utilities.
• A bull market versus a bear market.

I think it was the final statement that made me "invent" my own statement: Don't optimize an "uptrend" system using down-trending data.

• I think you're okay as long as you ignore the periods via your model. Don't just skip the dates manually. Your model will need to live in the real world so it will need to know how to identify and respond to conditions.
– Eric
Commented Mar 9, 2015 at 13:34