Matrix Notation for Regression Model

$$y=X\beta +u$$ $$y=\left[\begin{array}{c}{y}_1\\ y_2\\ ⋮\\ y_n\end{array}\right]X=\left[\begin{array}{cccc}1& x_{21}& \dots & X_{k1}\\ 1& x_{22}& \dots & X_{k2}\\ ⋮& ⋮& \ddots & ⋮\\ 1& x_{2n}& \dots & x_{kn}\end{array}\right]\beta =\left[\begin{array}{c}{\beta }_1\\ {\beta }_2\\ ⋮\\ {\beta }_k\end{array}\right]u=\left[\begin{array}{c}{u}_1\\ u_2\\ ⋮\\ u_n\end{array}\right]$$

So, $y_{(n\times 1)}$, $X_{(n\times k)}$, ${\beta }_{(k\times 1)}$ and $u_{(n\times 1)}$.

Frisch–Waugh–Lovell (FWL) theorem.

The Frisch-Waugh-Lovell (FWL) theorem states that in a linear regression model, the following two procedures yield identical estimates of a subset of coefficients:

1. Direct Regression: Regressing the dependent variable on all independent variables (including the subset of interest and other control variables).
2. Partialing Out:
   • Regress the dependent variable on the control variables.
   • Regress the subset of independent variables of interest on the same control variables.
   • Regress the residuals from the first regression on the residuals from the second regression. The resulting coefficients on these residuals will be the same as the coefficients obtained from the direct regression on the subset of variables of interest.

In simpler terms, the FWL theorem tells us that we can isolate the effect of a subset of variables by “partialing out” the influence of other variables. This is useful for understanding the unique contribution of specific variables in a complex model. It also simplifies calculations in some cases.

Basic Operations

Matrix operations

• Matrices must be confirmable for multiplication.
  • N X k and k X M