I have troubles estimating volatility (= standard deviation of log returns) when the data is re-sampled at different sample frequencies.
Problem
I have generated a time series data using a geometric Brownian motion. The original time series data is generated at an 1 hour interval for half a year:
$$ \begin{align} \mathrm{days} & = 365 / 2 = 182.5\,, \\ n & =182.5 \cdot 24 = 4380\,, \\ \Delta t & = \mathrm{days} / 365 / n=0.000114155\,. \end{align} $$
Assumptions:
- Trading is continuous throughout the year, so the number of trading days is 365.
- $t=1$ means 1 year (=365 trading days).
The orginal GBM time series $\{x_1 , \ldots , x_n\}$ is generated using the following parameters:
$$ \begin{align} \mu & = 0.5\,, \\ \sigma_\mathrm{1h} & = 0.8\,, \\ x_0 & = 1000\,, \\ \Delta t & = 0.000114155\,. \end{align} $$
Using this parameters, the daily and annual volatities are:
$$ \begin{align} \sigma_\mathrm{daily} & = \sigma_\mathrm{1h} \sqrt{24} = 3.92\,,\\ \sigma_\mathrm{annual} & = \sigma_\mathrm{daily} \sqrt{365} = 74.88\,. \\ \end{align} $$
Parameter estimation
Now I estimate the volatility using the following formulas:
$$ \begin{align} r_i & = \log{\frac{x_i}{x_{i-1}}} \qquad \textrm{log returns}\\ \hat{\mu} & = \frac{1}{n}\cdot\sum_{i=1}^n{r_i} \qquad \textrm{mean return}\\ S_\mu^2 & = \frac{1}{n-1}\sum_{i=1}^n{(r_i-\hat{\mu})^2} \qquad \textrm{sample variance of returns}\\ \hat{\sigma} & = \frac{S_\mu}{\sqrt{\Delta t}} \qquad \textrm{estimated volatility} \end{align} $$
Applying it to the original time series (1h interval) data, I get correct results:
$$ \begin{align} \hat{\sigma}_\mathrm{1h} & = 0.7970 \\ \hat{\sigma}_\mathrm{daily} & = 3.871 \\ \hat{\sigma}_\mathrm{annual} & = 73.959 \\ \end{align} $$
Parameter estimation from re-sampled data
However, when I re-sample the time series data into 1 day intervals (or any other interval), the estimated sigma $\hat\sigma$ stays the same (around 0.8), but since the frequency is different, $\Delta t$ is different and $\hat{\sigma}_\mathrm{daily}$ and $\hat{\sigma}_\mathrm{annual}$ is incorrect.
Re-sampling the time series to 1 day bins, gives
$$ \begin{align} n & = 183 \\ \Delta t & = \textrm{days} / 365 / n = 0.002725 \\ \hat{\sigma}_\mathrm{1d} & = 0.773 \\ \hat{\sigma}_\mathrm{daily} & = 0.775 \\ \hat{\sigma}_\mathrm{annual} & = 14.811 \\ \end{align} $$
I'm pretty sure is that the problem lies somewhere in the scaling of the volatility, but I can't figure out what exactly is wrong here.
The re-sampled (1 day bins) data do have a different time interval between each data point, so $\Delta t$ has to be different.
Implementation in Python
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(1018)
mu = 0.5
sigma = 0.8
x0 = 1000
start = pd.to_datetime("2021-01-01")
end = pd.to_datetime("2021-07-02 11:59.9999")
freq = "1H"
date_index=pd.date_range(start, end, freq=freq)
n = len(date_index)
dur = end-start
dur_days = (dur.days + dur.seconds / (3600 * 24))
dt = dur_days / 365 / n
print("n:", n)
print("dur_days: %.1f" % dur_days)
print("dt: %.8f" % dt)
x = np.exp((mu - sigma ** 2 / 2) * dt + sigma * np.random.normal(0, np.sqrt(dt), size=n).T)
x = x0 * x.cumprod(axis=0)
df0 = pd.DataFrame({"values": x}, index=date_index)
df = df0.copy()
real_daily_vola = sigma * np.sqrt(1 / dt / 365)
real_ann_vola = real_daily_vola * np.sqrt(365)
print("sample vola (=sigma): %.2f for freq %s" % (sigma, freq))
print("real daily vola: %.2f" % real_daily_vola)
print("real ann vola: %.2f" % real_ann_vola)
def estimate_vola(s):
n = len(s)
start = s.index[0]
end = s.index[-1]
dur = end - start
dur_days = dur.days + dur.seconds / (3600 * 24)
dt = dur_days / 365 / n
r = np.log(s / s.shift(1))
mu = np.nanmean(r)
s2 = 1 / (n-1) * ((r - mu)**2).sum()
est_sigma = np.sqrt(s2 / dt)
daily_vola = est_sigma * np.sqrt(1 / dt / 365)
ann_vola = daily_vola * np.sqrt(365)
return est_sigma, daily_vola, ann_vola
print("Estimate original vola: sigma=%.2f daily_vola=%.2f ann_vola=%.2f" % estimate_vola(df0["values"]))
# Re-sample to 1d interval
df = pd.DataFrame({"open": x, "high": x, "low": x, "close": x}, index=date_index)
df_1d = df.resample('1D').agg({'open': 'first', 'high': 'max', 'low': 'min', 'close': 'last'})
print("Re-sampled to 1d bins: sigma=%.2f daily_vola=%.2f ann_vola=%.2f" % estimate_vola(df_1d["close"]))
# Re-sample to 2d interval
df = pd.DataFrame({"open": x, "high": x, "low": x, "close": x}, index=date_index)
df_2d = df.resample('2D').agg({'open': 'first', 'high': 'max', 'low': 'min', 'close': 'last'})
print("Re-sampled to 2d bins: sigma=%.2f daily_vola=%.2f ann_vola=%.2f" % estimate_vola(df_2d["close"]))
Output
n: 4380
dur_days: 182.5
dt: 0.00011416
sample vola (=sigma): 0.80 for freq 1H
real daily vola: 3.92
real ann vola: 74.88
Estimate original vola: sigma=0.79 daily_vola=3.87 ann_vola=73.96
Re-sampled to 1d bins: sigma=0.77 daily_vola=0.78 ann_vola=14.81
Re-sampled to 2d bins: sigma=0.79 daily_vola=0.56 ann_vola=10.70
Note
The scaling factor 1 / dt / 365 in line
daily_vola = est_sigma * np.sqrt(1 / dt / 365)
to convert $\hat\sigma$ into the daily vola is correct, I think. It is:
| Re-sample freq | 1 / dt / 365 |
|---|---|
| 1D | 1 |
| 2D | 0.5 |
