Let's say I have the tracking error calculation for a portfolio:

enter image description here

How would I determine the N-obervations required for a statistically significant tracking error? Alternatively, how would I determine if the tracking error itself were statistically significant?

Some illustrative code here

import statsmodels.stats.moment_helpers as mh
import pandas as pd
import numpy as np
def generate_correlated_random_return_matrix(annual_means, annual_vols, corr, t_periods, n_samples, period_adjust=12.):
    Generates a return matrix from a multivariate random normal distribution.

        *annual_means*: An array of mean annual returns.

        *annual_vols*: An array of annual vols.

        *corr*: Correlation matrix. An example being:
            >>> [[1,0],[0,1]]

        *t_periods*: How many months would you like to simulate?

        n_samples**: How many times do you want to run this simulation?
    means = np.divide(annual_means, period_adjust)

    vols = np.divide(annual_vols, period_adjust ** .5)

    cov = np.asmatrix(mh.corr2cov(corr, vols), float)

    sim_array = np.random.multivariate_normal(means, cov, [n_samples, t_periods])

    return sim_array

te_tests = generate_correlated_random_return_matrix(annual_means=[.03,.03],annual_vols=[.1,.1],corr=[[1,.8],[.8,1]],t_periods=10000,n_samples=1)

df = pd.DataFrame(te_tests[0])

expanding_te = pd.expanding_std(df[0] - df[1])

mu = (df[0] - df[1]).std()

true_te = (df[0] - df[1]).std()

vol_of_expanding_TE = expanding_te.std()

z_score_of_TE_at_obs_N = ((expanding_te - true_te)/vol_of_expanding_TE).plot()

This I guess would give me a way to state the "measured TE is statistically indistinguishable from the TRUE TE", I suppose. Unsure if what I am using as standard deviation is correct, though.

  • 1
    $\begingroup$ @noob2 I agree. Some "noodling" type code in the updated Q above. It generally gets at the spirit of the problem. $\endgroup$
    – jason m
    Feb 4 '20 at 1:34
  • $\begingroup$ What is mh.corr2cov? $\endgroup$
    – Sandu Ursu
    Feb 4 '20 at 11:53
  • $\begingroup$ @SanduUrsu Sorry about that; Q updated. $\endgroup$
    – jason m
    Feb 4 '20 at 14:01

Suppose you have a portfolio that has some unknown real mean tracking error, $t$, relative to the benchmark, with some real variance $\sigma_t^2$.

You have a sampling process generated from your data that determines the estimated mean tracking error, $\hat{t}$ according to your formula, and of course you can derive an estimated variance in the tracking error also, $\hat{\sigma_t^2}$.

How can you assess the confidence in your dataset relative to the unknown real values?

1) You can run a non-parametric bootstrap sample: In this method you create multiple bootstrap sample datasets via resampling (with replacement) of the tracking errors from your dataset. Then you derive a confidence interval from the statistics of the bootstrap estimators.

As an example suppose you have 5 datapoints, 5 days worth of tracking errors:
[ 1, 2, 1, 3, 20 ], has a mean of 5.4 and variance of 67. How accurate or misleading might these estimators be? I perform 100 bootstrap samples (with replacement) and the resulting distribution I get of mean estimators is depicted:

enter image description here

Because of the small number of samples (and potential outlier) this is quite dramatic. I would suggest you cannot confidently assess you have an accurate estimator of the mean tracking error at 5.4. However with much larger N I think you will derive a fairly confident value.

For example, suppose I extend the dataset of tracking errors to 20 datapoints:
[1, 2, 1, 3, 20, 5, 3, 2, 8, 9, 4, 4, 7, 16, 2, 2, 2, 7] has a mean of 5.4. Now 100 bootstrap samples will yield the following distribution of mean estimators:

enter image description here

2) You can do the same process for a parametric bootstrap sample: where you assume the tracking error has some underlying distribution, and you can also do this for the variance estimator also.

In your case this might be more appropriate, since you have an underlying multivariate normal distribution.

In this case I would rephrase the question to what is the width of range of TE estimator values that is statistically significant given N takes different values.

For example suppose I create 20 parametric bootstrap samples for the cases where N is 3, 6, or 24 in each sample. I have a simple benchmark portfolio with weights [1,1] and a tracking portfolio of [0.9, 1.1]. I have simulated market movements with simple uniform distribution and no correlation and calculated the TE according to your formula. The distribution of TE estimators that I got looked like the following:

enter image description here

Clearly as N increases you have less variance in the estimators distributions and by performing this kind of analysis you can reference definitive confidence intervals that you are comfortable to work with. I.e. in this case it is statistically unlikely that the tracking error will be +- 0.2 from true value with N=24, but with N=3 there is a reasonable chance that will occur.

These methods form part of the field of computationally intensive statistics.

  • $\begingroup$ Let's assume, for my case, that we don't know the true mean or variance. Let's say I simulate the 10k returns to calc a "true" TE. Therefore, I know the true mean and variance of my data. If I then bootstrap some subset with replacement, what am I trying to find? Is it such that the mean of my sample is not statistically different than the mean and likewise for the variance? Do I just do this in a way such that I increase my sample size until I cannot differentiate? $\endgroup$
    – jason m
    Feb 4 '20 at 14:08
  • $\begingroup$ maybe my edit helps?? $\endgroup$
    – Attack68
    Feb 4 '20 at 14:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.