Best method to determine future success or to determine best linearity?

Question

Long time viewer, but first time poster, so excuse me if i'm in the wrong place please.

Anyway, I am working on a project that is pretty interesting. Through data mining, I am able to gather a ton of investment portfolios. Each portfolio has the obviously related statistics, including total revenue, total loss, resulting profit, and I can even get a daily break down of this for the past 3-4 years. (1200 days exactly).

I did a bit of excel manipulation, using the r2 pearson function and using a sharpe ratio (average profit/standard deviation), and I think I am on the right track because I was able to get a few results that were pretty damn consistent. First I break up the 1200 days into 20 day segments, and I get the net daily profit/loss. Then I accumulate the 60 points of 20-day intervals to get a growing summation. When you do this, you'll get some sort of line since you're adding up the individual intervals, and then I apply the 2 functions of r2 pearson and sharpe ratio to this line. My goal is to find portfolios that are linear in fashion, because in my eyes, if the portfolio has been linear for the past 1200 days, it should have a high chance of continued linearity.

So when you sort out my list of portfolios by highest sharpe ratio, I can choose one of the top portfolios and as you can see from the pic of the graph attached, it is somewhat consistent and linear as opposed to another portfolio that, when graphed, appears to be sinusoidal, or erratic with large jumps and dips.

My question is, can anyone give me more information on how valid my theory of using linearity for continued success is? Is there a different equation that I should be using to determine which portfolio I can invest with in the future? Is there something more capable of defining linearity of my portfolio graphs than the r2 function or the sharpe ratio?

I was also thinking of using a portfolio that ranks high in =(average sum)/(loss)) or choosing a portfolio that has the least amount of losses in it. Typically though, I have observed those having alot of flat areas when graphed, and I'm not sure that would be best for future success.

Thanks!!

Updated (1/25/2021) Below is a regression analysis done via excel using toolpak. I know the p-value is important, but is there anything in here that says this is terrible?

Answer 1

The problem with your question is phrase "continued success". Any formal test exploring linearity and upward direction would be based on the assumption that the future data are well-represented by the historical data. If this was the case, you could estimate model $$ \text{Portfolio} = \beta_0 + \beta_1 * \text{Day} + \beta_2 * \text{Day}^2 + \varepsilon. $$ Then you could use bootstrap sampling to show that

• $\beta_1$ is statistically significant and positive (upward direction),
• $\beta_2$ is non-significant (linearity over time).

The problem, of course, is that the past is not indicative of the future. Even if some crafty trading strategy is behind the overall portfolio performance, sooner or later this strategy is discovered or becomes less effective.

Answer 2

This won't be a brief one...

The potential pitfall with linearity (or log-linerarity) is best explained with two examples. There is a single (US equity) fund manager who was able to beat the S&P500 index sixteen years in a row. He was called Bill Miller; and he used to manage the LeggMason US Value fund. There aren't or have never been (given frequent replacement of failures by new attemptees) 2^16 fund managers (~65,000) out there to suggest this would be likely. But this isn't the point. The punchline is that Bill retired in 2009, after having just given up all of his accumulated 16 years of steady outperformance in 18 months through the GFC. He was "linear up" before a structural break down.

A less anecdotal version of the same would be consider the returns of call-overwriting strategies. Most of the time, these will enjoy a greater share of underlying market returns than the loss in market exposure, giving you more of the market upside for less of the volatility. The straight-line up appears cleaner. Except the saw-tooths down (equal and opposite in risk-adjusted terms most of the time) appear equally and oppositely more jagged. Hedge funds habitually have returns that share similar statistical properties. So the question is how much anyone's samples of returns contain more or less than the population of either regime.

But let's park such cynicism aside. We are looking at a time series in which we can be almost-certainly confident that it's non-randomly appreciating, even adjusting for any reasonable numeraire (like inflation and/or dividends etc.). It stands to reason that the more linear (or log-linear) it is, the better a standard regression fit would be. So the lower the standard error, the greater the precision around that fit, the greater the confidence in predicting its parameters.

Except here is where the rub lies. There are multiple different ways to create the charts you are looking at, each of which has different statistical properties, that allows you to infer different conclusions (even assuming the past continues into the future). It could be a trend (which is what regression analysis assumes). It might be a random walk with drift; or an "almost-random walk", ie an autoregressive process. It might even be a proper random walk with once-in-lifetime good luck ;-)

These can all appear very very similar; but have very different investment implications if that is the true process. And the barrage of statistical tests out there are all doing their best; but they do struggle to distinguish between Tweedle-Dee and Tweedle-Dum above.

The Durbin-Watson (DW) is a basic test for autocorrelation. But there is also eg Breusch-Godfrey, which doesn't assume (like DW) that any datapoint has to be related to its immediate neighbours. What this family of autocorrelation tests do is tell you that any deviations of datapoints from the regression line cease to be random. In which case, linearity ceases to be relevant or representative; and the usual assumptions and conclusions associated with regression cease to apply.

But hope is not lost. Your attractive time-series, if not linear, might be an autoregressive process, a random walk with drift, or an extremely lucky drunk just flipping heads over a good year or decade. The critical distinction between these is the existence of a so-called "unit root", ie is the beta of today's price =1 or <1 to yesterday's price. Which is what the Dickey-Fuller tests attempt to work out.

If it's autoregressive (ie beta to prior >0 as per DM, but <1 as DF), then you need an AR() model that includes the prior as an additional correcting input to your linear or log-linear model. Then the residuals should be random (or a million miles closer to such), and regression on these terms remains a valid best-guess to reality.

But if it's a random walk with drift, then you cannot regress; but are stuck in the classic mean-variance of Markowitz et al. A high mean with a low vol would remain attractive; but the trade-off is a risk-reward trade-off; and any kind of regression on price would be superfluous at best and misleading at worst.

The caution about linearity in general is that is you assume the analytical worst case, that there is no linearity, not even an autogression, and it is then just a random walk, then the analytical path of least resistance would be to calculate the Sharpe Ratio. About which a ton of papers have been critical in the extreme as a measure of risk.

It's hard to win in this quant game, really. Quant finance is a pick yourself out of the gutter, reboot, and try again, again, and again game like no other ;-)