This is a common problem (with backtesting in general); and one that is easy to lazily generalise; but difficult to pin down in a more forensic manner.
Beating buy-and-hold isn't a definitive test of whether your strategy is "good" or "bad"; but it does offer an intuitive and consistent alternative to measure it against. Measuring the ratio of your strategy against B&H tells you quickly when your strategy made/lost relative to B&H.
Optically, it's quite clear that the 2000-2020 backtest earns it keep by dodging the bear markets of the dotcom and GFC crashes. Those periods were flat in black and down in red. Black trumps Red. And it's equally optically clear that the reverse was true in the second 2015-2020 run. The flat black periods tended to be red-positive after the. If you measured the market (ie B&H) returns in the periods when your strategy was out of the market, I'd be very very surprised if they weren't negative before 2015 (maybe even 2013 or 2011); and positive after then.
Which might be suggestive that the payoffs associated with "buying the dip" changed between the two regimes. One can point fingers at the central banks and foam at the mouth ranting about QE; but one can never quite causally prove anything about its market impacts.
One can equally worry about backtests, like armies, always "trying to fight the last war", taking a Maginot Line to a tank fight. The point being that everyone's backtests now know about the GFC, the "risk-on,risk-off" regime that followed it, and the "liquidity-on, liqduity-off" regime that ensued after circa 2013. They're all optimising to the same limited sample.
I recall building a strategy like this circa 2005, that got bottled up into a range of structured products. Back then, all of definitions of what "disaster" looked like were heavily, heavily, biased to the 97-98 Asia/Russia/LTCM crisis, probably even more than to dotcom. However constructed, any model that flunked that scenario would be rejected out of hand without further examination. The "easy" way to do that was to "get out when things turn south" (like your strategy optically appears to do); but then "get back in when things turn so bad they can't really be that bad". Do that at -2 sigma, you made a lot of money with a lot of vol; at 3 sigma, you dodged most of the downturn with a huge roundtrip reduction in vol... Which is where most these strategies hit landmines in 2008. They bought the dip in Q4-08 after Lehmans, and oops. But if your model was identical, but configured to try to short the downswings, then you didn't buy the dip. When things got that bad, you sat in "I don't know" cash. Then you looked like a genius (selling on the way down before closing shorts near to the bottom). Even when everybody's model was virtually identical to all of the others! It was just configured to implement differently to different segments. Those segments being different in 2008 to 1998, as simple as that. Hero and villain alike were all collectively over-fitted to 1998 then.
Fast forward, and 2008 still plays the same role today as 1998 used to play before 2008. It is "the big one" that any model has to navigate. And just as 1998 biased everyone's reactions to 2008, everyone's memories of 2008 has the same potential to bias them today.
Proof of concept would be to optimise your strategy for the 2015-20 and see how it did during the dotcom and GFC episodes. I'll bet that the "genius" market-timing strategy of the last 5 years would have been taken to the cleaners then... because it won't be so biased by 2008 to cause it to ignore a very different market reaction function since then. Whether the reaction function is different because we all remember 2008 is, of course, unprovable.
But if you have a 2000-15 model that is neg-alpha in 2015-20; and a 2015-20 model that is neg-alpha in 2000-15, that would from basic first principles suggest that the market's reaction function was different between the two periods in question. Which is the central problem facing all market-timing strategies!
