Current industry standard for (active/passive) portfolio optimizations

Question

By reading multiple research papers online. I realized the current portfolio optimization (industry standards) involves building factor models, perform (conditional) value at risk optimizations, (covariance matrix shrinkage) and even robust portfolio optimization and (even though very few people cited, the method to impute price histories when different instruments have different inception dates). But the information is really very scattered, I wonder if anyone could suggest some list of current state of the art portfolio management/ active portfolio managements(trading signals) methods that I could read into to understand more with the goal of being able to implement state of the art methods by myself.

Answer 1

This is an important chapter in any investment management textbook. It is also poorly executed by the majority of money managers.

• Risk factor investment:
  • micro factors on the firm
  • macro factors on the sector, industry or index
  • price factors such as momentum, weightings, market cap and other price signals
• Asset allocation risk/return investment decision:
  • var-covar
  • expected shortfall
  • VaR
  • etc
• Portfolio construction:
  • return choices: e.g. floater or fixed rate bond + asset swap
  • asset-liability decisions: e.g2. AAA 4% jul 2067 or AAA 4.25% aug 2067
  • other idiosyncrasies
• Macroeconomic investment themes (alpha on time-varying investment), essentially hedge funds:
  • multi-asset (S.L. GARS, A.I. AIMS)
  • global macro fund
  • currency fund
  • statistical arb

Very often, a specialist investment will be made in funds with experience in each one of these sub-specialisations. A fund would not typically be expected to trade momentum strategies plus sector credit together.

Answer 2

I am a professor of finance who has spent his life working in the capital markets in operations, sales, compliance, and research. I would love to tell you about the existence of industry standards, but they do not exist. There is little improvement in the state of the art since the 1970's.

As a disclosure note, I am a strong critic of mean-variance finance arguing that there is a non-existence proof in statistics which excludes CAPM, Black-Scholes and Fama-French style models. I can find you 4,000 articles on just one anomaly in Ito calculus based methods. Indeed, almost all finance conferences are presentations on anomalies.

Let me give you the strands of thought, their advantages, and their disadvantages.

The first is the granddaddy of all financial methods, Graham and Dodd's fundamental analysis and margin of safety investing. This is also often called value investing. I teach a class in this and I am working on an economics paper arguing that Graham and Dodd's methods stochastically dominate most other methods. I say most because they do not deal with liquidity or liquidity costs and presume a very long horizon.

The advantage is that, with modification, you should be able to construct Kelly Criterion bets. As Kelly Criteria are log optimal in the absence of constraints and can readily be modified to included constraints, they should generally be used in the absence of a specific cost or utility function.

There are a few material disadvantages to Graham and Dodd methods. The first is that they depend upon data. They are not useful, in and of themselves, without lots of information about a firm. This could be supplemented by Bayesian analysis by forming a prior based on other firms, but that is shaky. They also ignore short-term decision making and judgments based on liquidity and liquidity costs. They are also not immune to false positives or false negatives, but without additional support also provide no diagnostics as to the rate of false discovery.

The next to appear in time is mean-variance finance. The key element of mean-variance finance is the existence of variance and the existence of a covariance matrix. Again, as a disclosure note, I have argued that there does not exist a variance in the distributions involved so you may consider me biased here.

These methods are heavily used with a wide variety of methods to try and shore them up. In particular, you mentioned shrinkage methods, factor models and VaR as well as robust methods. You missed things such as heteroskedasticity based estimation and testing unless that is what you meant by robust methods. You also excluded things such as ANN to back into mean-variance finance.

The advantage is that the tools provide precise allocations. They also provide for social acceptance because of the two Nobel prizes. The list of disadvantages is quite long.

First, even if the models were true, it has been shown that these models can always be stochastically dominated. Indeed, they lack the statistical property known as coherence which implies that money should never be gambled on them. The models are constructed under the assumption that the parameters are perfectly known. What is missing from that is that if they are not, then, by theorem, no estimators exist for the models. Finally, they have generated a huge literature of their faults. Indeed, for a professor under publish-or-perish, they provide a fertile ground to talk about something that does not work. They are easy pickings. It is the low hanging fruit.

Do note that I am including, probably improperly, factor and PCA models such as the APT and Fama-French. None of these models are supportable in the sense that they have a validation study that shows they work out-of-sample, they are all dominated models and they lack coherence. The impropriety is that they see themselves as something else, but they share all the math issues with traditional mean-variance finance.

The next class of models beginning with Mandelbrot in 1963 and running with Eugene Fama until the 1970s are the heavy-tailed distribution models. These models lack a first moment, and so expectations cannot exist, nor can a covariance matrix.

Again, as a disclosure note, I fit inside this group. There cannot exist an admissible, computable, unbiased non-Bayesian estimator for this class of distributions. The distributions lack a sufficient statistic forcing all non-Bayesian estimators to lose information and are truncated at -100% strongly biasing non-Bayesian estimators.

The advantage of these methods, when done carefully, is that they work and generate very narrow interval estimates. They allow 98,000% returns as part of nature, which is fortunate as this has happened, but there is a small mountain to climb in Bayesian decision theory on how to use them. You are assured an admissible and coherent estimator if you are very careful. Indeed, you are assured of using all available information if you use the predictive distribution. There is also a clear methodology for scoring predictions. Since they are admissible, they dominate all other estimators including shrinkage estimators.

The disadvantages are plentiful. Bayesian methods are computationally intense. They require you to perform numerical integration, possibly in high dimensions, without the existence of analytic solutions. Furthermore, Bayesian hypotheses are combinatoric. Whereas $y=\beta_1x_1+\beta_2x_2+\beta_3x_3+\beta_0$ requires one F-test in Frequentist statistics, it requires eight sets of hypothesis tests for all possible combinations of variables, unless you can logically exclude some. This requires eight sets of integrations. It doesn't take long for a Bayesian solution to become computationally prohibitive.

They are also difficult to get published. There are no p-values. There is no null hypothesis. Furthermore, they can be difficult to sell as they are subjective models. Two researchers with different information will get different results.

The next in time are behavioral methods. There are two problems with behavioral methods that are also their advantage. They are descriptive and not prescriptive. They describe behavior well, but may not describe equilibrium behavior.

You should probably investigate and use behavioral methods unless you believe that you are not human and so lack the problems human investors have. With that said, they may not matter in equilibrium. The issue is that many observed behaviors are contrary to the systemic data.

Let me provide an example. It has been argued, and I believe to be true, that humans engage in hyperbolic discounting rather than exponential discounting. That may be true, however, as future value is present value times one plus the rate, that assures that the grand economy will grow at an exponential rate. This means that any other discounting is not actually possible, at the margin, though it may be caused by many individuals hyperbolically discounting in a very heterogeneous manner.

The same thing is true regarding bias. Bayes theorem says all actors should be biased estimators if their decision making is coherent in the statistical sense, except under generalized Bayes rules. However, if real people used generalized Bayes rules who were also not toddlers or academics then you would likely have to worry about them. You would use a generalized rule when you could honestly say "I know nothing at all about what I am doing."

As an example, if you were to use a generalized Bayes rule when walking on the edge of a skyscraper then you would consider walking out into thin air as equivalent to walking toward the center of the building towards safety.

If you use an ordinary estimator that is Bayes, then you are assured bias with a probability of one hundred percent, by theorem. That said, this is useless knowledge. Imagine that the economy took action $X$ which is $Y$ in magnitude. Also, imagine all the actors were slightly biased above and below on both measures. The supply and demand curves still intersected at $(X,Y)$ and any unbiased predictions would still work. Individual actors being biased doesn't imply anything about an equilibrium other than that some may be more surprised than others.

You should read heavily on behavioral methods, but as a description of the roadblocks, you will create for yourself and not as a prescription for your behavior.

I am going to bypass econophysics because it falls under the heavy-tailed category.

If you were my student and you had access to serious datasets, then I would give you the 1943 book Security Analysis by Graham and Dodd, I would get you a copy of Decision Theory by Parmigiani, I would assign to you the literature on bankruptcy and mergers, and I would require an extensive background in numerical methods to support all the terrible integrations you would be doing. Finally, I would have you read the original article on the Kelly Criterion and assign readings on optimization, including optimization that would not meet the Kelly Criterion such as when someone had a utility function where there was a need for a sufficient amount of funds, but where they found higher returns of no value. There are people for whom enough is enough and that is a different class of problem. You can think of them as concave to a point and then flat.

EDIT

I looked again at my answer and realized I left two strands of thought out of the discussion that need addressed and studied by you, but which fall outside the categories above.

The first is financial intermediation and this includes models such as the Diamond and Dybvig model for banking and models of broker-dealers and I would take you at least back to the Treynor model though there are obviously newer and better models of microstructure and institutions.

The second is to model liquidity, which is somewhat a separate issue from the models of intermediation, though they cannot be separated. I would look at Abbott's chapter in The Valuation Handbook on discount factors for marketability and liquidity.