Kalman Filter Implementation using PyKalman

Question

I am trying to apply a simple Kalman filter to pair trading. My underlying stock pair is cointegrated with no constant term. Can someone kindly advise if i am going on the right track with the Kalman filter?

from pykalman import KalmanFilter
import numpy as np
import pandas as pd

# Log prices of stock_1 and stock_2
stock_1_price = np.log(df_combined[f"Close_{stock_1}"])  # independent variable
stock_2_price = np.log(df_combined[f"Close_{stock_2}"])  # dependent variable

# Transition matrix for beta (1x1 identity matrix because we're only estimating beta)
transition_matrix = np.eye(1)  # 1x1 identity matrix

# Observation and transition covariances
observation_covariance = 1  # Observation noise, can be tuned or optimized
transition_covariance = np.array([[0.001]])  # Process noise covariance (small to keep beta stable)

# Initial estimates
initial_state_mean = np.array([0])  # Initial guess for beta
initial_state_covariance = np.array([[1]])  # Initial covariance

# Set up the Kalman Filter
kf = KalmanFilter(
    transition_matrices=transition_matrix,
    initial_state_mean=initial_state_mean,
    initial_state_covariance=initial_state_covariance,
    observation_covariance=observation_covariance,
    transition_covariance=transition_covariance
)

state_means = np.zeros(len(stock_2_price))  # beta over time
state_covariances = np.zeros(len(stock_2_price))

state_mean = initial_state_mean
state_covariance = initial_state_covariance

# Run the Kalman filter manually in a loop
for t in range(len(stock_2_price)):
    observation_matrix = np.array([[stock_1_price[t]]])  # stock_1_price for beta
    state_mean, state_covariance = kf.filter_update(
        state_mean,
        state_covariance,
        observation_matrix=observation_matrix,
        observation=stock_2_price[t]
    )
    state_means[t] = state_mean[0]
    state_covariances[t] = state_covariance[0, 0]

# Extract the hedge ratios (beta)
hedge_ratios = state_means  # beta values over time

# Calculate the spread using the estimated hedge ratios
kalman_spread = stock_2_price - hedge_ratios * stock_1_price
plt.plot(kalman_spread)
```

Answer 1

A few considerations:

In your code, the observation_matrix is a 1x1 matrix based on stock_1_price[t], which is incorrect. For the Kalman filter to work correctly, you need to ensure that the observation_matrix and observation are correctly sized. In your case, since you're estimating a single beta, the observation matrix should be a 1-dimensional array rather than a 2D matrix.

The kf.filter_update method is typically used to update the state estimate in each iteration. However, this method is meant to be used within the Kalman filter's internal loop, and you should use the kf.filter method for the entire time series data.

You should ensure that the dimensions and initializations of state_mean and state_covariance match the expected dimensions of your state vector.