How to interpolate gaps in a time series using closely related time series?

Question

I am trying to construct a daily time series of prices and returns for some large universe of securities. However, all I have available are a monthly time series of the prices/returns (as well as other characteristics) of the individual securities, a daily time series of a market-value-weighted index of all securities, and weekly time series of various sub-indices.

The constructed time series will ultimately be used to estimate parameters of a more general model, such as the probability of a security's issuer taking some action (e.g. refinancing their debt) as a function of the security's price. Therefore I feel it is not important to maintain causality. The issuer presumably knows the true price when taking the action, even though I do not, and I need to construct a best guess as to what the price was given everything I know today.

Note: it is not possible to obtain higher frequency data at the individual security level, either because the securities themselves do not trade that often, or because (AFAIK) nobody collects the data. The goal is to interpolate a reasonable-looking set of daily prices and returns based on all available information. Any advice on how to carry out this estimation would be appreciated.

I have some of my own ideas, which I may share after a while, but right now I'm still in the exploratory phase and I'm looking for some additional inspiration.

Just to make it clear what I mean by way of example, suppose I wanted to find the daily prices of all 1500 stocks in the S&P 1500, but all I had were monthly prices for the stocks, weekly prices for the 10 GICS sector indices and for the large cap 500, mid cap 400, and small cap 600, and daily prices for the S&P 1500 as a whole.

The purpose, in that example, would be to fit a model of announcements of share buybacks and secondary offerings based on interpolated valuation metrics.

UPDATE: One answer suggested applying the Expectation-Maximization algorithm. As far as I can tell, EM is not applicable to this problem. Applying EM to price, one gets a sawtooth-pattern where the filled values are on a different plane from the known values. I can't figure out a way to apply EM to returns, since I'm not missing any monthly returns, and I'm missing all daily/weekly returns for the individual securities.

Answer 1

You must apply the E-M algorithm to an invariant (time-homogenous i.i.d. variable) such as log-returns -- not prices.

The key to the E-M is is the simplifying assumption that the invariant (namely the distribution of returns) as well as the distribution of missings are i.i.d. Prices do not obey this property. The trick of assuming an i.i.d. invariant and then proceeding to impute originates with Little and Rubin (1987).

In your case, clearly the distribution of missings is not random however. The literature refers to this case as "Not Missing at Random". You can do some tests or rely on theory to determine whether assuming the distributions are "Missing at Random" or "Missing Completely at Random" (MCAR) is valid.

The bibliography of the paper Multiple Imputation for Missing Data (2003) cites the key papers in this area.

EDIT:

I read your update and noticed that you have only monthly returns not daily/weekly.

Here's one approach where you can still make the E-M method work.

At the monthly level, you have the security returns alongside the returns for the various indices and sub-indices. Measure the monthly covariance of the log returns and mean monthly log-returns of the various assets. Now project the monthly covariance to a daily level (simply divide both parameters by # of trading days in a month). You have daily returns for the S&P 1500. Now fill in the missing entries by replacing the missing values with their expected value conditional on the observations of daily prices for the S&P 1500 (using the E-M algorithm). The final step is to convert the log-returns back to arithmetic-returns.

Note that you are assuming that the correlation structure is stable over the estimation period. Your output would consist of normally-distributed well-behaved returns. These are the usual defects of the E-M approach.

Answer 2

This answer only deals with obtaining higher frequency data from low frequency data. The second method is taken from the draft of a master thesis of a friend of mine, i.e. most of this is taken from an unpublished source.

Jones (1998) propose an algorithm to this using something similar to Gibbs sampler to get the most likely parameter values for a diffusion given the data, in short:

1. Choose some initial values for the parameters $\phi$ and the unobserved data $X^u$.
2. Redraw the unobserved paths, or “bridges”, in between the observed data and fill in any latent variables by cycling, point by point, through the elements of $X^u$.
3. Draw new parameters conditional on the augmented data set.
4. Go back to (2) or terminate the Markov chain if convergence has been determined.

This model can then be used to obtain observations at any frequency using an Euler scheme. See Jones (1998) for further details. My friend modifies this method as follows:

1. Choose initial parameters
2. Generate data conditional on observations and current parameters by cyclic MH
3. Optimize the likelihood for the chain generated in step (2)
4. Save optimal parameters of step (3), use them as input for step (2) and repeat
5. After a burn-in period average the saved parameters to obtain the final estimates

My friend argues:

The original algorithm of Jones employs a Gibbs sampler instead of EM. Rather than optimizing in step (3), Jones draws new parameters out of some prior distribution and accepts or rejects these draws conditional on the chain generated in step (2). It is however very unlikely, to accept a certain draw, especially for higher dimensional parameter spaces. Furthermore computationally it is more efficient to optimize than to draw hundreds of chains for a single change of parameters. When using Gibbs sampling it is also impossible to impose restrictions on a combination of variables, such as forcing the distribution to be unimodal. A disadvantage is that we loose the favorable properties of Bayesian inference.

Unfortunately I can't provide you a full answer. I'm not sure if it has been done yet (publicly).

Answer 3

I think you need to say something about what you want to do with the "filled in" series. If you're interested in statistical properties, the usual technique is maximum likelihood estimation using the EM algorithm. That gives you something like a completion of the missing values, but only in the context of the statistic being extracted -- that is, you're "not allowed" to use the filled-in values for anything else, because they are just determined by the condition that you get the correct maximum likelihood value when you do conventional statistical calcs (e.g., pairwise covariances) on the completed series.

Answer 4

The basic rule to keep causality during resampling/interpolation of financial data is not to use information from future. You need to use stepwise interpolation by "dragging" the last known information along new samples until the next monthly update. You must know when exactly these monthly values where sampled/calculated. This guarantees causality, but not correctness of the data in the gap points, because interpolation does not add any information to the original monthly data.

Answer 5

A weekly series of low-frequency (monthly) variables is obtained using an interpolation, or “adjustment” with respect to a related series. The interpolation of a time-series by means of a related series involves two steps: choosing the “benchmark” series, and then interpolating the wanted series using the related series. The related series is chosen so that its movements at the high-frequency intervals are highly correlated with the given series.

For example, unanticipated inflation data is extracted using monthly CPI data. Therefore, the series has an original monthly frequency. Using a weekly related time series, the weekly resampling of unanticipated inflation is performed by superimposing the intra-month movements of the related series on the month-to-month movement of the monthly unanticipated inflation factor, using a technique to eliminate any difference between the month-to-month movements of the two series.

Let's look at this example. Let: - UNIF: be the monthly series of unanticipated inflation. This is the original time-serie. - x: be the related time-series to unanticipated inflation, with a weekly frequency. - UNIF*: be the interpolated time-series (the one we want to derive) with aweekly frequency.

Here is the methodology: