0
$\begingroup$

Let's say a have a price series for a share for the year 2000. On the 27th of July 2000, there is a missing value represented by NA. This was not a holiday or any other non trading day as other shares have values on that same day.

Now when calculating returns, what is the best approach to use? Do I fill the NA with the previous observation or do I leave it as is? The former method creates a return of 0 on the 27th of July while the latter creates two NA values and another one on the 28th of July. Which is the more appropriate method in this case?

$\endgroup$
  • $\begingroup$ The answer would depend on the purpose and sample length you have. If you have sufficient enough observations, dropping one is negligible. Alternatively, you might also want to consider replacing NA with a median of (some windows of) the sample. $\endgroup$ – Evgenii Sep 22 at 20:01
  • $\begingroup$ It depends. For many purposes filling in the 27 July price with the previous price is acceptable. For a more sophisticated approach you would need a statistical model that predicts the price (based for ex on what the market as a whole did that day). In most cases I use the former approach. I never use a median, it could give a misleading return. $\endgroup$ – Alex C Sep 22 at 20:07
  • 1
    $\begingroup$ It's subjective how you handle this - but probably the most used method is 'forward-filling' -- just take the most recently observed value. $\endgroup$ – Jared Sep 22 at 20:07
2
$\begingroup$

There isn't a single answer to this question. It strongly depends on your goals and why it is missing. If you have a long enough time-series, you will find large numbers of missing data points. The NYSE used to maintain a post for companies that did not trade weekly not so long ago.

However, unless you have cause to believe there was a reason for it to be omitted other than that no trade happened on that day, there are three rather simple solutions (remembering that a simple solution in statistics is usually non-existent).

The first is to note that if an allocation is $p\times{q}$, then it isn't $p$ that is missing, it is that $q=0$. If you were working with allocations rather than prices, then you eliminate the problem because $pq=0$. However, very few people work with allocations, mostly those who work with liquidity related questions. The other issue is that it will look like a jump and/or cause a division by zero error.

The second is to treat the missing price as a parameter rather than as data. You should only do this if you believe the value is missing because of a reporting error rather than because of the lack of a trade. It will require that you use a Bayesian method. Bayesian methods not only have natural ways to deal with missing data that are well understood, but they also do not disturb parameter estimates. For example, putting in the median of the interim days' prices disturbs the estimation of both the center of location and the scale parameter.

Imagine a very simple model, for illustration purposes of $p_{t+1}=\beta{p_t}+\alpha+\epsilon_{t+1}$. Also imagine that you are missing only one observation, observation $k$. You can have many missing ones as the process is the same for many as for one. Having one simplifies notation.

If no observations were missing, then you would have to construct a prior probability density for $\beta,\alpha$ and $\sigma,$ the scale parameter of the model. In addition, it would be possible there were other parameters such as $\theta$, but we will ignore anything else. The solution would be $$\Pr(\beta;\alpha;\sigma|p_1\dots{p}_T).$$

With a missing observation, you would construct a prior distribution for $\beta,\alpha,\sigma,$ and $p_k$. The nice thing is that you can use $p_1\dots{p_{k-1}}\cup{p_{k+1}}\dots{p}_T,$ to form the prior distribution regarding the location of $p_k$. You would be solving $$\Pr(\beta;\alpha;\sigma;p_k|p_1\dots{p_{k-1}}\cup{p_{k+1}}\dots{p}_T).$$

If you would then marginalize out the distribution of $p_k$ what would happen is that the impact of $p_k$ being missing would have no adverse impact on either inference or point estimators of the other parameters. If you have not used a Bayesian method, then I strongly recommend getting the assistance of a professional statistician. It is not a tool for novices. Although it is straightforward to understand, it can be tricky to do, particularly if the relationships are not independent as would be the case in a time series.

The third one can be done with a Bayesian methodology, but I do not believe it could be done within the method of maximum likelihood. It has no Frequentist counterpart at all.

With the third one, omitted observations are treated as a form of data by altering the likelihood function. The absence of an observation is information. Let us go back to the AR(1) function above.

$p_{t}=\beta{p}_{t-1}+\alpha+\epsilon_{t}$ and $p_{t+1}=\beta{p}_t+\alpha+\epsilon_{t+1}$ with observation $p_t$ being non-existent because no trade happened. You can estimate $p_{t+1}$ by having a contingent likelihood function of $p_{t+1}=\beta^2{p}_{t-1}+\beta\alpha+\alpha+\epsilon_{t+1}$. This also has the nice advantage in that you could resize your scale matrix (covariance matrix if normal) based on observations that are present.

Let us imagine that you have a covariance matrix of size $N$ when all prices have an observation. If one were missing the row and column of the missing observation would not generate an observation, and the matrix would be $(N-1)\times(N-1)$ instead of $N\times{N}$ for that one day.

If you are missing a price for one security on July 27, and you are calculating returns as $\frac{p_{27}}{p_{26}}-1$ and $\frac{p_{28}}{p_{27}}-1$ you have two choices.

You could change the likelihood function and solve $$\sqrt{\frac{p_{28}}{p_{26}}}-1,$$ or you could leave both returns blank.

If you are not treating the returns as a time series, that is $r_{t+1}=f(r_t)+\varepsilon_{t+1}$, then you will have Frequentist, method of maximum likelihood, and Bayesian methods. If you are treating it as a time series, then you would have a Bayesian method, as above, by changing the likelihood. There is probably no disciplined method of maximum likelihood solution. What is often done in Frequentist methods is to impute the price either by averaging the surrounding prices or by forecasting the value from prior prices.

The reason I generally oppose the interpolation/forecasting solution is that its impact is not neutral on parameter estimates. If it is one observation of thousands of observations, then it is sensible to use interpolation or forecasting because the effect will be small enough to ignore. The issue happens with a large shift in prices and/or many missing values. Most prices move slightly from day to day, but equity prices can have large shifts. I am avoiding the word "jumps" because they have a technical meaning that I do not want to use here because it presumes a model.

Imagine on July 27 there was a large marketwide movement of prices, but your security didn't trade at all. Maybe it is a generally illiquid security, and on downward shifts, there is a tendency for them to lose volume. Interpolating an intermediate price could easily make it look like it is an unusually less volatile security than the market. In that case, it is not an innocent omission. For such securities, it is a case of market failure. It is highly informative, and forecasting or interpolation would be misleading if your model includes other securities.

If you were doing this for a course assignment and the number of omissions are small, I might interpolate or forecast and explain it away. If you are gambling with real money, you should strongly consider a Bayesian solution because it will incorporate the full information set of your entire model.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.