Michaud's Resampled Efficient Frontier - Out of Sample Simulation Testing

Question

I will be putting ALL my account points on bounty to whoever answers this question [if your answer is crap but it's the only answer, you're getting the 165 points]. You will have to wait 2 days or so from now to receive the points. Yes, all $165$ points!

I purchased Michaud & Michaud's 1998 book called "Effective Asset Management". In it they detail their Resampled Efficient Frontier (REF) methodology. In chapter 6 they present an out of sample simulation test. I'm trying to replicate this test but am having difficulty understanding what they mean because I am not good with a completely wordy explanation of how to do a simulation study. I'll place snippets of their simulation instructions into code and what I think I have to do to replicate the simulation into enumeration/dot-points.

My question is: Have I interpreted their instructions correctly?

---

In a simulation study, the referee is assumed to know the true set of risk-returns for the assets.

1) I have $\mu$ and $\Omega$, these are considered "true"/population.

The referee does not tell investors the true values but provides a set of Monte Carlo simulated returns consistent with the true risks and returns. In the base case data set, each simulation consists of 18 years of monthly returns and represents a possible out-of-sample realization of the true values of the optimization parameters.

2) I simulate $\mathbf{r}\sim MVN(\mu,\Omega)$, which will be a $T\textrm{x}(\textrm{Number of assets})$ matrix. Here, they are setting the matrix dimensions to be $(18 years*12 months)\textrm{x}(\textrm{Number of assets})$.

Each set of simulated returns results in an estimate (with estimation error) of the optimization parameters and an MV efficient frontier. Each MV efficient frontier and set of estimated optimization parameters defines an RE optimized frontier.

3) This is what I'm most confused about. I'll try to guess at what they're saying here. Here's my guess: Estimate $\hat{\mu}$ and $\hat{\Omega}$ based on the simulated $\mathbf{r}$. Then (i) do MVO optimization over $\hat{\mu}$ and $\hat{\Omega}$, and (ii) Do REF optimization by simulating $500$ futures from $\hat{\mu}$ and $\hat{\Omega}$, resulting in $\hat{\mathbf{r}}_i\sim MVN(\hat{\mu},\hat{\Omega})$,$i \in \{1,...,500\}$. Using $\hat{\mathbf{r}}_i$ to find $\hat{\hat{\mu}}_i$ and $\hat{\hat{\Omega}}_i$, and then find the REF based on these $500$ estimates. This will result in two frontiers, one for (i) and one for (ii).

This process is repeated many times.

4) Repeat steps 2) to 3) $N$ times, at each step storing both the MVO and REF frontier weights matrices that you calculate in 3) (it's a weights matrix because it's a bunch of vectors at each point along the frontier). This will result in $N$ matrices of weights for MVO, where each $n \in \{1,...,N\{$ is the results of one of the simulations 2)$\rightarrow$3). The same for the REF.

In each of the simulations of MV and RE optimized frontiers, the referee uses the true risk-return values to score the actual risks and returns of the optimized portfolios.

5) For each $n \in \{1,...,N\}$, we have weights matrices $_nw_{RE}$ and $_nw_{MV}$. Taking $_nw_{{RE},i}$ as one vector in the matrix of weights $_nw_{RE}$, find $_nw_{{RE},i}' \mu$ and $_nw_{{RE},i}' \Omega_nw_{{RE},i}$ for each $i$. This will result in a graph in $(\sigma,\mathbb{E}r)$ space. Do the same for $_nw_{{MV},i} \forall i \forall n$.

The averaged results of the simulation study are displayed in Exhibit 6.3. The upper dotted curves display the in-sample averaged MV and RE frontiers that were submitted to the referee for scoring. The higher dotted curve is the MV efficient frontier; the lower dotted curve is the REF. The portfolios are plotted based on the simulated risks and returns. However, the referee knows the true risks and returns for each simulated optimized portfolio. The bottom solid curves in Exhibit 6.3 display the average of the true, out-of-sample, risks and returns of the optimized portfolios. The higher solid curve represents the RE optimized results, the lower solid curve the Markowitz optimized results.

6) Massive confusion. What are these "average results"? I'm guessing they do the unweighted mean of each of the $(\sigma,\mathbb{E}r)$ results (for all $N$) in step 5) for both methodologies and then plot both frontiers. However what's this OOS/in-sample business? Does steps 1) to 5) result in the OOS frontiers? Is the in-sample frontier the standard REF and MVO calculated from the true $\mu$ and $\Omega$?

---

Here's an alternative explanation from Michaud and Michaud, in 2008 paper "ESTIMATION ERROR AND PORTFOLIO OPTIMIZATION: A RESAMPLING SOLUTION".

Following Jobson and Korkie (1981) and Michaud (1998, Ch. 6), we perform a simulation test to compare RE vs. MV optimization. In a simulation test, a referee is assumed to know the true values of asset risks and returns. The 20 stock risk-return data, shown in the Appendix, serves as the “truth” in our simulation experiments. The referee creates a simulated history and provides returns that are statistically consistent with the true risk-return estimates. These returns can either be thought of as historical observations of a stationary return distribution, or a number of noisy estimates of next period’s return. The Markowitz and RE investors compute their efficient portfolios based on the referee’s supplied returns. The referee uses the true risk-return values to score the optimized portfolios. Figure 4 gives the average of the results after many simulation tests.

The curves displayed in Figure 4 represent the averaged results from the simulation test. The left-hand panel displays the average MV and RE efficient frontiers computed from the referee’s returns, the portfolios that were submitted to the referee for scoring. The higher (red) dotted curve is the MV efficient frontier; the lower (blue) dotted curve is the REF. The left-hand panel represents what the Markowitz and RE investors see on average given the referee’s data. The right-hand panel of Figure 4 illustrates the average results of how the submitted efficient frontier portfolios performed when the referee applied the true risk-returns. The higher (blue) solid curve represents the RE optimizer results; the lower (red) solid curve the Markowitz optimizer results. The right-hand panel of Figure 4 shows that the RE optimizer, on average, achieves roughly the same return with less risk, or alternatively more return with the same level of risk, relative to the Markowitz optimizer.

Answer 1

Before answering your questions directly i would like to briefly restate the idea of the resampled efficent frontier:

One of the problem with classical mean variance optimization is (even if the multivariate normal assumption holds) that you cant estimate $\mu$ and $\Omega$ (which is usually denoted as $\Sigma$) exactly. Thats why you incur estimation errors $\hat{\mu}-\mu$ and $\hat{\Omega}-\Omega$ when you deviate from the "population means". This error can change your portfolio dramatically (since MVO is not very robust in general). So the idea is to estimate $\mu$ and $\Omega$ to end up with $\hat{\mu}$ and $\hat{\Omega}$ and sample from the $\text{MVN}(\hat{\mu},\hat{\Omega})$ distribution. From all these samples, calculate the efficient frontier and the optimal portfolio. The RE frontier and RE optimal portfolio will be the mean of those.

This method is a bootstrapping method (where you strangely gain information for adding uncertainty).

Another nice way to think of it is (and here i quote the paper you mentioned): "Every simulated MV efficient frontier [...] is the right way to invest given a set of inputs. But the inputs are highly uncertain. How should an investor deal with portfolio optimality uncertainty?"

And the answer is: One should use the REF to obtain a sub-optimal (for one estimated set of parameters) portfolio which is more prone to estimation error and therefore should result in a better portfolio than from estimation alone. (look at Figure 4 in the paper you mentioned!)

As for your questions, you got everything correct up to step 3.):

3.) I don't know what you mean by futures. I would call them assets. What you should do is you take your values $\hat{\mu}$ and $\hat{\Omega}$ and calculate the RE frontier. To do that you draw lots of samples from $\text{MVN}(\hat{\mu},\hat{\Omega})$, do MV optimization on each of them and compute the mean (both of the optimized pf and the frontiers).

4.) As you said, repeat the steps several times.

5.) You are right here. Just "score" all the portfolios (i think you meant $\Omega$ instead of $\Omega_n$).

6.) The plot shows the average in-sample and out-of-sample efficient frontiers. in-sample: You take all MV efficient frontiers for $\hat{\mu}$ and $\hat{\Omega}$ and build the average. That gives you the "Classical in-sample" curve. Computing the average over the REFs for the same parameters gives you the "REF in-sample" efficient frontiers. In the chart this means that if you consider the average of your sampled parameters as the true population parameter, the MV efficient frontier is always abvoe the resampled efficient frontier. out-of-sample: You take all calculated MV and REF weights but now you calculate the efficient frontier with respect to the true population parameters $\mu$ and $\Omega$. Of course (since they are optimal for another parameter set) this efficient frontiers are lower then their in-sample counterparts. Now comes the important point: The MV efficient frontier is now inferior to the REF. Thats what the whole point of the bootstrapping technique was.

Hope that enlightened you.