# Is my data fittet to be significant?

I am new to this forum and hope for some help. I have a dataset of returns. I cannot tell where these come from, but let's assume they come from a trading algorithm of stock prices. The returns are measured weekly and represent a portfolio of different stocks.

Changing some parameters (when to buy and when to sell) in the algorithm will "randomly" change the returns. I would call this random because I change the parameters until the overall return at the end of the year becomes maximum. But when positive and negative weekly returns are generated will be completely random to the parameters.

So, after having found the optimum, I test the weekly returns for significance.

Now I have some questions to this procedure:

1) The most important questions is: Is it ok to fit the parameters until I find [both,] the highest yearly return [that result from statistical significant weekly returns]? Is the procedure of finding a significant weekly return ok? If not, what else could I do?

2) What is the typical method of testing (weekly) returns? My guess was regressing the weekly return against its constant and testing for significance. Another idea is that I would create a bankroll and a time variable. So I can test if the growth of the bankroll is significantly increasing.

I would be glad if you could help me and link some journal articles that explain the field of returns more in detail. Thank you very much!

UPDATE, in order to clarify my problem a little more: The procedure is that I first vary some parameters. This will lead to different yearly returns. My variable of interest ist the weekly return. This will also vary with a change in the parameters. Then I change my parameters such that the yearly return becomes maximum. Then I run a regression on the weekly returns against their constants. The problem here is that the weekly returns from one stock are r=random[-1,x], where x > 0. So they are either -1 or x. Next I combine either the weekly returns of two different stocks into a portfolio or I combine two consecutive weeks. Value of -1 are now only the exceptions. The new portfolio will return weekly a value of p=random[y] where y can be any rational number. After this, I run tests on this weekly portfolio return. Note that I do only try to optimize the yearly return. And I stabilize variance of the weekly return by putting them into "pseudo" random portfolio. Note that the weekly returns can be interpreted as time series, but they are not really finacial time series. This is because the return don't come from stocks. I just assumed this to illustrate the case. So problems with time varying variance should not occur.

PS: I already asked this question in statsexchange but did not receive an answer.

## 1 Answer

The question you ask is in fact about what people in machine learning call overfitting:

• the more you choose your "metaparameters" to provide high returns on your sample of days
• the less you can trust them to reproduce out of this sample.

There is not a lot you can do to prevent this except read a lot to understand overfitting and be very careful. Two main directions:

• for data scientists: read Vapnik's book: you will learn here the effect of nonlinearity on overfitting.
• for an exconometric approach, read Harvey's paper: you will learn how test a lot of simple models can be enough to overfit your data.
• Thanks for this information! Now let's suppose I do not run "too many" tests. I just try to find the maximum return after one year by not looking to much at the weekly returns. The outcomes are either -1 or +x. So my idea was to combine stocks such that the weekly return is between -infinite and + infinite. In detail: I combine weekly returns either by adding more stocks or by putting two weeks together in order to increase diversification (which should reduce the variance in this portfolio weekly return). Is that procedure still ok? – Econ Aug 8 '17 at 6:26