Counterintuitive time varying Beta with Kalman filter

Question

If you're used to play with R, you'll enjoy the following reproducible code:

# =================================================== #
# An example of state-space monitor via Kalman filter #
# =================================================== #

op <- par(no.readonly = TRUE)
Sys.setenv(TZ = 'UTC')

# Contents:

# 1. Installing packages
# 2. Loading packages
# 3. Custom functions
# 4. Downloading and preparing data
# 5. Cross-Betas Kalman filtering

# *********************************
# 1. Installing packages
# *********************************

install.packages('KFAS')
install.packages('latticeExtra')
install.packages('quantmod')

# *********************************
# 2. Loading packages
# *********************************

require(compiler)
require(latticeExtra)
require(KFAS)
require(quantmod)

# *********************************
# 3. Custom functions
# *********************************

# This function returns the time varying state-space representation 
# parameters of a linear model which represents y ~ X

Kalman.beta <- cmpfun(function(y, X)
{
  model <- regSSM(y = y, X = cbind(1, X), H = NA, Q = diag(NA, 2))
  object <- fitSSM(inits = rep(0, 3), model = model)$model
      KFAS <- KFS(object = object)
      alpha.beta <- xts(t(KFAS$alphahat), index(y))
  colnames(alpha.beta) <- rep(paste(colnames(y), 'vs' , colnames(X)), 2)
  return(alpha.beta)
})  

# *********************************
# 4. Downloading and preparing data
# *********************************

env <- new.env()
Symbols <- c('SPY', 'QQQ', 'XLF', 'TLT')
getSymbols(Symbols = Symbols, env = env, from = '1950-01-01')
args <- eapply(env = env, FUN = function(x){ClCl(x)})
X <- na.omit(do.call(what = merge, args = args))
colnames(X) <- Symbols
xyplot(X)

# *********************************
# 5. Cross-Betas Kalman filtering
# *********************************

Betas <- NULL
k <- 0

for(i in 1:ncol(X))
{
  for(j in (1:ncol(X))[-i])
  {
    k <- k + 1
    Betas[[k]] <- Kalman.beta(y = X[,i], X = X[,j])[,2]
  }
}

Beta.matrix <- do.call(what = merge, args = Betas)
colnames(Beta.matrix) <- gsub(pattern = '.', fixed = TRUE, 
                              x = colnames(Beta.matrix), replacement = ' ')
xyplot(Beta.matrix, superpose = FALSE, auto.key = FALSE,
       main = '')

What does this code do? It basically uses Kalman filter to estimate time varying $\beta_{t}$ of each asset against each other and plot them.

What's the matter with that?

If you use a simple linear regression model to estimate $\beta$ constant over time you will see it often happens, as instance, that $\beta_{t}<1<\beta$ or $\beta_{t}>0>\beta$ for the most of the time series... which is really counterintuitive!

How could SPY be negatively correlated with QQQ while it's quite obvious they are strongly correlated and $\beta \approx 1$? And so on...

How would you explain this?

Is there anything wrong with my code?

Answer 1

This is definitely not a Kalman filter's issue: if you replace this line of code

args <- eapply(env = env, FUN = function(x){ClCl(x)})

with this one

args <- eapply(env = env, FUN = function(x){ClCl(x)})[Symbols]

eapply() will keep the order of the original Yahoo query from quantmod. You can check and you will see each $\beta_{t}$ matches about the $\beta$ from simple OLS linear regression CAPM-like.