For S&P, or any time series for that matter. When doing analysis on the distribution of the yearly returns, should I be looking at 1) the daily year over year values, 2) pick some starting point like December 31st of year YYYY and then look at YoY, or 3) something else.

My intuition is that for 1) you are artificially creating many "YoY" values that look the same because the previous days YoY will not change much given a one day move, but at the same time 2) seems a little arbitrary because youre just selecting a single point as a reference point (you could have chosen first of year or middle of year) and in fact 1) contains 2)

Doing both for S&P returns from Yahoo from 1950s doesnt seem like it shows a big difference in terms of mean or vol parameters (the daily YoY has wider min/max as you would expect):

> smry
          dailyYoY     yearly
Min.    -0.6699000 -0.4499000
1st Qu. -0.0120600 -0.0061320
Median   0.0937500  0.0834800
Mean     0.0724200  0.0736000
3rd Qu.  0.1783000  0.1681000
Max.     0.5222000  0.3274000
Std Dev  0.1555526  0.1487778

It seems like one could argue that the yearly is just a sampled version of the dailyYoY, but which should I be using? I'm leaning more towards dailyYoY, because the other method ignores information I already know

enter image description here

  • $\begingroup$ In the left graph you can see some smoothing of the density by averaging over many starting dates. Looks prettier. However the things you are averaging are not independent of each other, so the statistical properties of the LHS graph are complicated, and the vertical axis is misleading (oversetimates the d.o.f).. As you say, there are advantages and disadvantages. $\endgroup$
    – Alex C
    Commented Jun 25, 2017 at 22:37
  • $\begingroup$ Personally for visualizing the density I prefer dailyYoY. But for inference I prefer yearly because the observations are independent. (To estimate a confidence interval for the mean, for ex. I would use yearly). $\endgroup$
    – Alex C
    Commented Jun 25, 2017 at 22:50
  • $\begingroup$ But using only the yearly points seem too arbitrary and it ignores more information you already have? $\endgroup$
    – qwer
    Commented Jun 25, 2017 at 23:38
  • 1
    $\begingroup$ Just picking one point in a year would be inefficient (in the statistical terminology). There are methods that deal with overlapping data. You might search for this term e.g. on Cross Validated and/or more broadly. $\endgroup$ Commented Jun 26, 2017 at 19:04

1 Answer 1


In my view there isn't a good answer to this question; the daily method introduces autocorrelation between your returns and the yoy samples leave you with little data.

I would let whatever you choose (and I'll mention some of the other things that you could think about below) mainly be guided by what you are trying to achieve with you analysis, e.g. do you care about the mean, standard deviation, worst loss, or still something else? Do you believe that recent returns are more relevant than those from 50 years ago?

So some ideas for what you could do beyond what you mentioned (and depending on your goals these could be reasonable or highly inappropriate):

As hinted at in the comments on the question, for the daily points there are some methods available to adjust e.g. the estimator for the standard deviation of yearly returns to correct for the autocorrelation. So that's one way to go. If there isn't an analytical correction, you could come up with one based on simulation from what you consider to be an appropriate idealized distribution.

Another way to go could be bootstrapping. If daily bootstrapping isnt useful for your application, a good trade-off might be to divide your data in weekly or monthly blocks and just sample 52 cq 12 of those.

If you're really interested in the tails, you could also look at something like extreme value theory for estimating returns.

Yet another way to go, if you're interested in crises (as people looking at very long time series often are) could be to model the data as bring emitted from a hidden Markov model where markets can be in a few states (e.g. {normal, crisis, recovery}) and estimate return distributions and transition probabilities from them and then sample from you this model.

Yet another alternative could be looking at what distribution fits your daily data well and simulate how that aggregates from daily to yearly for certain percentiles of the distribution.

  • $\begingroup$ Thanks, follow up questions: 1) On bootstrapping, what do you mean by 52 cq 12? 2) For the actual way to "adjust e.g. the estimator for the standard deviation of yearly returns to correct for the autocorrelation", do you have something to point me to read? $\endgroup$
    – qwer
    Commented Jul 4, 2017 at 21:28
  • $\begingroup$ 1) I was referring to the idea that you could boostrap from weekly or montly returns rather than from daily returns, to allow for some non-gaussian aggregation effects. 2) Hodges and Tompkins, the sampling properties of volatility cones is well-known. This also looks good based on title/abstract, but I don't have access to it: repub.eur.nl/pub/2180 $\endgroup$
    – Bram
    Commented Jul 5, 2017 at 6:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.