0
$\begingroup$

The coefficient of regression $\beta$ is often expressed as:

$\beta = (X^TX)^{-1}X^Ty$

I came across the notation below. Can someone help me visualize how the summation of column vectors $x_i$ is equivalent to the matrix notation?

$X^TX = \sum_{i=1}^{n}(x_ix_i^T)$

$\endgroup$
3
  • $\begingroup$ What do you mean by how does it manifest? $X = [x_1 … x_i … x_n]$ is the matrix containing the values of the independent variables in regression. In finance, $x_i$ would typically be the returns of a set of indices for a specific period (e.g. yesterday’s S&P 500 return), and a 1 for any intercept term. $y$ is a vector containing the values of the dependent variables in each period. $X^{T}X = \Sigma^{n}_{1} x_i x^T_j$ are just two ways of writing the same calculation. $\endgroup$ Commented Apr 10, 2018 at 12:51
  • $\begingroup$ I could not figure out (visualize) how the summation equals the Matrix. $\endgroup$
    – whisperer
    Commented Apr 10, 2018 at 15:34
  • 2
    $\begingroup$ It's useful to be comfortable with both notations. My go-to is the matrix notation, but sometimes the other notation is useful. $\endgroup$
    – John
    Commented Apr 10, 2018 at 21:42

1 Answer 1

2
$\begingroup$

In econometrics it is usually simpler to use a small example. I also have a lot of issues in visualizing those notations and small examples always help me. Let's make an example:

$ X= \begin{bmatrix} 1 & 5 \\ 3 & 7 \end{bmatrix}$

Therefore: $ X^T X= \begin{bmatrix} 10 & 26 \\ 26 & 74 \end{bmatrix}$

Now two vectors: $x_1 = \begin{bmatrix} 1 \\ 5 \end{bmatrix}$ and $x_2 = \begin{bmatrix} 3 \\ 7 \end{bmatrix}$

So:

$x_1 x_1^T = \begin{bmatrix} 1 & 5\\ 5 & 25 \end{bmatrix}$ and $x_2 x_2^T = \begin{bmatrix} 9 & 21\\ 21 & 49 \end{bmatrix}$

Add both togheter: $x_1 x_1^T + x_2 x_2^T = \begin{bmatrix} 10 & 26\\ 26 & 74 \end{bmatrix}$

$\endgroup$
1
  • $\begingroup$ This example is good. Understanding dimensions is also helpful. In general $X$ has nr rows and nc columns. Note that $x_1,x_2,\cdots,x_{nr}$ are each of size nc, i.e. they are read off the rows of $X$. $X^T X$ is of size nc by nc, as is each $x_i x_i^T$. $\endgroup$
    – Alex C
    Commented Apr 10, 2018 at 19:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.