The coefficient of regression $\beta$ is often expressed as:

$\beta = (X^TX)^{-1}X^Ty$

I came across the notation below. Can someone help me visualize how the summation of column vectors $x_i$ is equivalent to the matrix notation?

$X^TX = \sum_{i=1}^{n}(x_ix_i^T)$

  • $\begingroup$ What do you mean by how does it manifest? $X = [x_1 … x_i … x_n]$ is the matrix containing the values of the independent variables in regression. In finance, $x_i$ would typically be the returns of a set of indices for a specific period (e.g. yesterday’s S&P 500 return), and a 1 for any intercept term. $y$ is a vector containing the values of the dependent variables in each period. $X^{T}X = \Sigma^{n}_{1} x_i x^T_j$ are just two ways of writing the same calculation. $\endgroup$ Apr 10 '18 at 12:51
  • $\begingroup$ I could not figure out (visualize) how the summation equals the Matrix. $\endgroup$
    – whisperer
    Apr 10 '18 at 15:34
  • 2
    $\begingroup$ It's useful to be comfortable with both notations. My go-to is the matrix notation, but sometimes the other notation is useful. $\endgroup$
    – John
    Apr 10 '18 at 21:42

In econometrics it is usually simpler to use a small example. I also have a lot of issues in visualizing those notations and small examples always help me. Let's make an example:

$ X= \begin{bmatrix} 1 & 5 \\ 3 & 7 \end{bmatrix}$

Therefore: $ X^T X= \begin{bmatrix} 10 & 26 \\ 26 & 74 \end{bmatrix}$

Now two vectors: $x_1 = \begin{bmatrix} 1 \\ 5 \end{bmatrix}$ and $x_2 = \begin{bmatrix} 3 \\ 7 \end{bmatrix}$


$x_1 x_1^T = \begin{bmatrix} 1 & 5\\ 5 & 25 \end{bmatrix}$ and $x_2 x_2^T = \begin{bmatrix} 9 & 21\\ 21 & 49 \end{bmatrix}$

Add both togheter: $x_1 x_1^T + x_2 x_2^T = \begin{bmatrix} 10 & 26\\ 26 & 74 \end{bmatrix}$

  • $\begingroup$ This example is good. Understanding dimensions is also helpful. In general $X$ has nr rows and nc columns. Note that $x_1,x_2,\cdots,x_{nr}$ are each of size nc, i.e. they are read off the rows of $X$. $X^T X$ is of size nc by nc, as is each $x_i x_i^T$. $\endgroup$
    – Alex C
    Apr 10 '18 at 19:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.