# Tag Info

## Hot answers tagged regression

29

Because of: The (extreme) dominance of noise over signal The prevalence of non-repeating patterns (many of which we know are not going to repeat) A pathetic sample size for cross-validation Regime changes due to exogenous events. These are typically in the cross-val window which makes it even worse. (GFC, financial integration, trade law changes, interest ...

12

1. Determine Factors Economically, the use of factor models can be either motivated using the ICAPM or the APT. Although there are some theoretical differences between the model, for empirical and practical work these differences are irrelevant. In the end, both models stipulate that returns and expected returns are linear functions of the factors: $$... 11 The regression requires orthogonalization of factors. However, we need to maintain the interpretation of factors (so PCA and Factor Analysis are out). Also, we could apply an iterative method (indeed this is very common practice) but this will bias the factor loadings on the sequence of factors. Best approach is that of Klein and Chow in their paper ... 10 A few thoughts. Yes, your return series are autocorrelated (i.e., stocks don't exactly follow a random walk), so you should use Newey-West standard errors. If you do this as a univariate regression$$R_{i,t} = \alpha_i + \beta_i R_{j,t-1} + \epsilon_{i,t}$$then there's almost certainly an omitted variable inside \epsilon that is moving both R_i and ... 10 I don't have much to add, but wanted to address the "price of risk" question. APT is kind of "economics"-free and tries to price assets without the utility maximization required in CAPM/ICAPM. Ross's APT observes that groups of assets move together (e.g., tech stocks) and that is the risk you're bearing because the idiosyncratic risk, like the firing of ... 9 The R^2s are usually close to zero for single stock regressions. The big R^2s that a lot of asset pricing research shows is by forming portfolios. Forming portfolios cancels a lot of the idiosyncratic returns, which has a smoothing effect. The R^2s should be low here, although I don't see any in the paper for you to compare. This probably means they ... 8 Jennifer Bender of MSCI Barra has a paper from 2007 entitled: To Beta or Not to Beta: A Comparison of Historical Versus Fundamental Betas for Hedging Market Risk She deals specifically and exclusively with which method is superior for hedging long-only portfolios. Not surprisingly, she finds that Barra's approach is better. She tests long-only and ... 7 It appears that you are re-running the regression with each new data point. Instead, you should use an update/online formula (see an excellent answer by the famous Dr. Huber at stats.se). You can find an implementation in the R package biglm. If it doesn't have all the features you need (no windowing out of old data) you can at least adapt it and use it ... 7 Regression analysis, as a minimization of the sum of squared errors, does not require normality of the error term. The requirements are that errors are homoscedastic and uncorrelated. And these are the fundamental assumptions (together with exogeneity). Then estimators are unbiased, optimal (exhibit the minimum variance within the class of unbiased ... 6 I basically agree with @John, let me expand: We want to model y using a simple linear model, the most basic setup is$$ y = c + \mathbf{X}\beta $$with y the N observations, c a constant, \mathbf{X} the N \times M matrix of regressors and \beta a M-dimensional vector of coefficients. This model has M parameters, the elements of \beta. ... 6 Have you considered fitting ARIMA with exogenous regressors model? Linear regression with autocorrelated errors might be appropriate. R can do this with the arima() function via specifying the xreg argument. 6 I believe that beta will be the covariance of the factor with the underlying asset. Is this correct? Close, it's the covariance divided by the variance of the factor. $$\beta_{f,a} = \frac{\sigma_{f,a}}{\sigma^2_f}$$ Also how is the return attributable to a specific factor calculated? Is there a single way this is done ... 5 Since you mention beta, I assume you're familiar with the capital asset pricing model (CAPM). The concept is that an asset's expected returns are linearly correlated with the market's returns. Of course, there are other ways "normalize" returns, as you put it. We can extend CAPM with Fama-French, which adds market cap and relative value to the equation. ... 5 If the equation satisfies all the assumptions of OLS, particularly homoscedasticity and no autocorrelation in the errors, then the expected return for the equation you laid out is E[r_{future}|r_{history},x_{news}]=\alpha+\beta_1r_{history}+\beta_2x_{news}+\beta_3r_{history}*x_{news} If the unconditional expected return is zero (as is likely to be ... 5 Couple points I like to make: There exists no reliable model that can even predict future price returns (risk premiums, excess returns, whatever you want to call it) beyond a year, run as fast as you can if you hear from someone who claims he can predict risk premiums 10 years out, whether reliably or not. It makes zero sense and clearly comes from either ... 5 The following paper (and the references given within) focuses on the practical aspects of implementation of factor-based investing and gives an overarching framework for the more technical answers here: Practical Considerations for Factor-Based Asset Allocation by Kang, X. (Standard & Poor's), Ung, D. (Chartered Alternative Investment Analyst ... 5 There is no a "yes/no answer" to that question. Generally Kalman Filter tends to be better than linear regression, but everything depends on the data which you have, how you calibrate your model. I expect that you have used some library for estimating linear regression parameters. Now you need to think how will you "tune" Kalman filter - the constants ... 5 Then for each month t, you run a cross-section regression: r_{i,t} = \lambda_0 + \hat{\beta}_i {\lambda}_t + \alpha_{i,t} Where: \hat{\beta}_i \equiv [\beta_{i, MktRf}, \beta_{i, SMB}, \beta_{i, HML}]', is a vector of the coefficients estimated on the first step. What you are looking for is to estimate the vector of \hat{\lambda}_t \equiv ... 5 The two step Fama-Macbeth regression works as follows: First, run a cross sectional regression in each period. I believe that you want to estimate risk premia for each of the Fama and French factors. Therefore you run:$$r_{i,t} = \lambda_{t,MKT} \hat{\beta}_{i,MKT}+\lambda_{t,HML} \hat{\beta}_{i,HML}+\lambda_{t,SMB} \hat{\beta}_{i,SMB}+ \alpha_{i,t} \quad ...

4

This question was ultimately answered on Cross Validated Here are a couple of articles that deal with this subject: Britten-Jones and Neuberger, Improved inference and estimation in regression with overlapping observations Harri & Brorsen, The Overlapping Data Problem

4

Generally we use models that go so far out in a comparative sense, not as an absolute decision. You are definitely do some good reading but I believe you are thinking about these models in the wrong way. I think (and correct me if I'm wrong) you are looking at creating or finding the perfect "crystal ball" model that will predict returns/risk premiums etc. ...

4

This isn't exactly what I would call advanced but running each regression on a separate core in a parallel foreach loop would help http://cran.r-project.org/web/packages/foreach/foreach.pdf

4

Note that you can understand the $\Delta$ as an "operator" acting on $r$. So just act on $r$ twice: $$\Delta^2 r_t = r_t - 2 r_{t-1} + r_{t-2}.$$ In fact if you write the $r$ as a vector, $r = (r_1, r_2, \ldots, r_N)$, then $\Delta$ is an $N\times N$ matrix with elements $\Delta_{i,j} = \delta_{i,j} - \delta_{i-1,j}$. The AR(2) model can be written as ...

4

I was going to comment but it turned out to be quite elaborate. My experience with certain AI/ML methods is that they're not deterministic. Take RBM for instance, a very wide-spread paradigm. To train such a machine you have two approaches, backpropagation or Kullback-Leibler divergence. Both require you to initialise the machine to a random state. And ...

4

It's probably because of the strong long-standing statistical underpinnings in economics and econometrics, and overall, risk prediction. For example, look at current research with fat-tail distributions and calculations for Expected Tail Loss (ETL), etc. These studies fit Student's t, Normal, Stable, and Pareto probability distributions to data and report ...

4

I was just like you when I started out: I had learned a lot about machine learning (mainly neural networks and genetic algorithms/programming) and used it heavily. I also had learned about classic statistics but not nearly as much as about ML. The problem with ML is - as I see it today - that you are often taking a sledgehammer to crack a nut, meaning: ...

4

Time Series Factor modelling is a very good and practical manual to building time series factor models. FactorAnalytics is a very good R package that allows you to fit timeseries, fundamental and statistical factor models. A good reference to factor models would be Chapter 15 of this book.

3

If $\sum_{i=1}^k \alpha_i<1$, then you could just leave the remainder of the portfolio in cash. If $\sum_{i=1}^k \alpha_i>1$, that means you will have to take on some leverage in order to minimize tracking error. If you have a leverage constraint, then you can run this as a quadratic program with bounds on your coefficients. A regression should give ...

3

What you are talking about is called regression using fractional polynomials and it has its merits. The canonical reference is this one: Regression Using Fractional Polynomials of Continuous Covariates: Parsimonious Parametric Modelling by Royston and Altman (1994) From the abstract: The relationship between a response variable and one or more ...

3

When trying to predict returns, I think you should never look at in-sample statistics like R-squared. Only look at out-of-sample prediction results. Cross validation is a useful tool in at least the initial phase of modelling. In addition to over-enthusiasm, in-sample statistics easily lead to overfitting: ...

Only top voted, non community-wiki answers of a minimum length are eligible