I've been given some data (it's financial tick data) and I want to predict based on some observed variables whether the next move will be up, down or unchanged.
So I have been trying to use multinomial logistic regression, this is my first time doing logistic regression so I want to check that I have done this correctly and that my results look reasonable.
Right now I am doing 3 bi-variate logistic regressions.
So I code the data such that I have three new time series labelled up, down and unchanged. These are generated by testing whether the move n steps ahead was up, down or unchanged in the original series and adding a one to the appropriate array and making all other entries zero.
I then do a bi-variate logistic regression of these up, down and unchanged arrays against the regressors individually.
I can then calculate the probability of each using the transformation:
$$\textrm{prob} = 1/(1+\exp[-Bx])$$
where $\beta$ are the betas from the bi-variate logistic regressions. And $x$ is the value of the regressors.
This gives me the probability of up, down and unchanged.
I then simply compare if $\textrm{probability of UP} > \textrm{probability of Down}$ if so the model predicts up and vice versa.
Q.1) Is my methodology correct? Right now I am doing all calculations on the price series (not the returns series)?
Q.2) When I test this accuracy in sample I am getting 70% accuracy in sample (for both up and down moves)? Is that a reasonable test score in sample?
Q.3) The model probability for unchanged is very low typically around 14%. So unchanged is never selected (because the probabilities of up down moves are always much larger). However unchanged is the most commonly observed change with an unconditional probability of 91%. Is there a way I can correct the model so that unchanged is forecast accurately by the model.
Update: Here is the code unfortunately I am getting differences between 2 variable regression results and the 3 variable results!
Possible error between two: 2 variable regression run using mnrfit() and an equivalent 3 variable version on asset returns. The returns have been classified as positive, negative or flat. The logistic regression is then run on the rtns vs the classified returns (this is a simple test to check that the regression functions as intended). When I do this for the 2 variable version i.e. Up rtns v everything else, the regression gives an estimate of the probability that a 0 return is not an up return of 88%. As the return size is increased the probability that it is positive increases eventually converging to 1 (as you would expect). Also as increasingly negative rtns are put into the logistic regression model the probability that the rtn is positive goes to zero. The same is true for the 2 Variable estimate of Down returns v everything else. The figures are similar to those above but with the signs of the returns reversed.
Now when I run the 3 variable version. Things initially look OK: when given a return of zero it estimates the probability that it is zero to be 86% with a prob of a down move =6.4% and an up move =7.6% so very similar to the 2 variable case. Moreover when larger and larger returns are entered the probability that it is a positive return converges to 1 as you would expect but when you put in larger and larger negative returns the probability that the return is negative converges to zero while the probability it is equal to zero increases to 1!!! Which is clearly wrong.
Data1 = LoadMat_SHFE_Range(Contract1, StartDate, EndDate);
rtn=(Data1.Mid(2:end)-Data1.Mid(1:end-1))./(Data1.Mid(1:end-1));
NStep=0;
Up=nan(length(rtn),1);
Down=nan(length(rtn),1);
Flat=nan(length(rtn),1);
RtnClass=nan(length(rtn),1);
for i=1:length(rtn)-NStep
if(rtn(i+NStep)>0)
Up(i)=2;
Down(i)=1;
Flat(i)=1;
elseif(rtn(i+NStep)<0)
Up(i)=1;
Down(i)=2;
Flat(i)=1;
elseif(rtn(i+NStep)==0)
Up(i)=1;
Down(i)=1;
Flat(i)=2;
end
end
[BUp,dev,stats] = mnrfit(rtn,Up);
MatProbUp = mnrval(BUp,0.1);
[BDown,dev,stats] = mnrfit(rtn,Down);
MatProbDown = mnrval(BDown,0.1);
for i=1:length(rtn)
if(rtn(i)>0)
RtnClass(i)=3;
elseif(rtn(i)<0)
RtnClass(i)=2;
elseif(rtn(i)==0)
RtnClass(i)=1;
end
end
[BM,dev,stats] = mnrfit(rtn,RtnClass,'model','ordinal');
[pihat,dlow,hi] = mnrval(BM,0,stats,'model','ordinal');