Frisch–Waugh–Lovell theorem explained

In econometrics, the Frisch–Waugh–Lovell (FWL) theorem is named after the econometricians Ragnar Frisch, Frederick V. Waugh, and Michael C. Lovell.^{[1] [2] [3]}

The Frisch–Waugh–Lovell theorem states that if the regression we are concerned with is expressed in terms of two separate sets of predictor variables:

Y = X _{1} \beta _{1} + X _{2} \beta _{2} + u

where

X _{1} and X _{2} are matrices, \beta _{1} and \beta _{2} are vectors (and u is the error term), then the estimate of \beta _{2} will be the same as the estimate of it from a modified regression of the form:M _{X _{1}} Y = M _{X _{1}} X _{2} \beta _{2} + M _{X _{1}} u,

where

M _{X _{1}} projects onto the orthogonal complement of the image of the projection matrix X _{1(X} ^{T} ^{1} X ^{-1} _{1 )} X ^{T} _{1} . Equivalently, M_X1 projects onto the orthogonal complement of the column space of X₁. Specifically,M _{X _{1}} = I - X _{1(X} ^{T} ^{1} X ^{-1} _{1 )} X ^{T} _{1} ,

and this particular orthogonal projection matrix is known as the residual maker matrix or annihilator matrix.^{[4] [5]}

The vector $M\_\; Y$ is the vector of residuals from regression of $Y$ on the columns of $X\_1$.

The most relevant consequence of the theorem is that the parameters in $\backslash beta\_2$ do not apply to $X\_2$ but to $M\_\; X\_2$, that is: the part of $X\_2$ uncorrelated with $X\_1$. This is the basis for understanding the contribution of each single variable to a multivariate regression (see, for instance, Ch. 13 in ^[6]).

The theorem also implies that the secondary regression used for obtaining

M _{X _{1}} is unnecessary when the predictor variables are uncorrelated: using projection matrices to make the explanatory variables orthogonal to each other will lead to the same results as running the regression with all non-orthogonal explanators included.

Moreover, the standard errors from the partial regression equal those from the full regression.^[7]

History

The origin of the theorem is uncertain, but it was well-established in the realm of linear regression before the Frisch and Waugh paper. George Udny Yule's comprehensive analysis of partial regressions, published in 1907, included the theorem in section 9 on page 184.^[8] Yule emphasized the theorem's importance for understanding multiple and partial regression and correlation coefficients, as mentioned in section 10 of the same paper.

By 1933, Yule's findings were generally recognized, thanks in part to the detailed discussion of partial correlation and the introduction of his innovative notation in 1907. The theorem, later associated with Frisch, Waugh, and Lovell, was also included in chapter 10 of Yule's successful statistics textbook, first published in 1911. The book reached its tenth edition by 1932.^[9]

In a 1931 paper co-authored with Mudgett, Frisch cited Yule's results.^[10] Yule's formulas for partial regressions were quoted and explicitly attributed to him in order to rectify a misquotation by another author. Although Yule was not explicitly mentioned in the 1933 paper by Frisch and Waugh, they utilized the notation for partial regression coefficients initially introduced by Yule in 1907, which was widely accepted by 1933.

In 1963, Lovell published a proof^[11] considered more straightforward and intuitive. In recognition, people generally add his name to the theorem name.

Further reading

• Book: Russell . Davidson . James G. . MacKinnon . Estimation and Inference in Econometrics . New York . Oxford University Press . 1993 . 0-19-506011-3 . 19–24 . none .
• Book: Russell . Davidson . James G. . MacKinnon . Econometric Theory and Methods . limited . New York . Oxford University Press . 2004 . 62–75 . 0-19-512372-7. none .
• Book: Trevor . Hastie . Trevor Hastie . Robert . Tibshirani . Robert Tibshirani . Jerome . Friedman . Jerome H. Friedman . The Elements of Statistical Learning : Data Mining, Inference, and Prediction . New York . Springer . 2nd . 2017 . 978-0-387-84857-0 . Multiple Regression from Simple Univariate Regression . 52–55 . https://web.stanford.edu/~hastie/Papers/ESLII.pdf#page=71. none .
• Book: Ruud, P. A. . 2000 . An Introduction to Classical Econometric Theory . New York . Oxford University Press . 0-19-511164-8 . 54–60 . none .
• Book: Stachurski, John . A Primer in Econometric Theory . MIT Press . 2016 . 311–314 . 9780262337465 . none .

Notes and References

1. Ragnar . Frisch . Frederick V. . Waugh . Partial Time Regressions as Compared with Individual Trends . . 1 . 4 . 1933 . 387–401 . 10.2307/1907330 . 1907330 .
2. Lovell . M. . 1963 . Seasonal Adjustment of Economic Time Series and Multiple Regression Analysis . . 58 . 304 . 993–1010 . 10.1080/01621459.1963.10480682 .
3. Lovell . M. . 2008 . A Simple Proof of the FWL Theorem . . 39 . 1 . 88–91 . 10.3200/JECE.39.1.88-91 . 154907484 .
4. Book: Hayashi, Fumio . Fumio Hayashi . Econometrics . Princeton . Princeton University Press . 2000 . 0-691-01018-8 . 18–19 .
5. Book: Davidson, James . Econometric Theory . Malden . Blackwell . 2000 . 0-631-21584-0. 7 .
6. Book: F. . Mosteller . J. W.. Tukey . Data Analysis and Regression a Second Course in Statistics . Addison-Wesley . 1977 .
7. Peng . Ding . 2021 . The Frisch--Waugh--Lovell theorem for standard errors . Statistics and Probability Letters . 168 . 108945.
8. Yule . George Udny . 1907 . On the Theory of Correlation for any Number of Variables, Treated by a New System of Notation . Proceedings of the Royal Society A . 79 . 529 . 182–193 . 10.1098/rspa.1907.0028. 2027/coo.31924081088423 . free .
9. Book: Yule, George Udny . An Introduction to the Theory of Statistics 10th edition . Charles Griffin &Co . 1932 . London.
10. Frisch . Ragnar . Mudgett . B. D. . 1931 . Statistical Correlation and the Theory of Cluster Types . Journal of the American Statistical Association . 21 . 176 . 375–392. 10.1080/01621459.1931.10502225 .
11. Lovell . M. . 1963 . Seasonal Adjustment of Economic Time Series and Multiple Regression Analysis . . 58 . 304 . 993–1010 . 10.1080/01621459.1963.10480682.