OLS Assumptions, Violations, and Diagnostics
Consider the linear regression model $Y = X\beta + \epsilon$, estimated by ordinary least squares (OLS).
- State the five classical OLS assumptions (Gauss-Markov plus normality).
- For each assumption, explain what goes wrong when it is violated -- specifically, does it affect bias, efficiency, or the validity of standard errors and hypothesis tests?
- For each violation, name a diagnostic test to detect it and a practical remedy.
Hints
- Organize your answer by what the violation breaks: bias vs. efficiency vs. inference. The Gauss-Markov theorem tells you OLS is BLUE -- know which violations break the B (Best), the L (Linear), or the U (Unbiased).
- Exogeneity ($E[\epsilon|X]=0$) is the only assumption whose violation biases $\hat{\beta}$ itself. All other violations affect standard errors or efficiency but leave point estimates consistent.
- For each assumption, pair the diagnostic test with the remedy: homoscedasticity -- Breusch-Pagan test, fix with White robust SE; autocorrelation -- Durbin-Watson, fix with Newey-West; multicollinearity -- VIF, fix with regularization.
Worked Solution
How to Think About It: The Gauss-Markov theorem says OLS is BLUE -- Best Linear Unbiased Estimator -- under its assumptions. But 'best' and 'unbiased' can break independently. Violations that bias $\hat{\beta}$ are the most dangerous (you get the wrong answer). Violations that only inflate standard errors are less severe (your point estimates are fine, but your t-stats are wrong). Know which is which.
Key Insight: Bias in $\hat{\beta}$ comes almost exclusively from Assumption 3 (exogeneity) being violated. All other violations mainly affect efficiency or inference, not the coefficient estimates themselves.
The Assumptions and Their Violations:
1. Linearity: $E[Y|X] = X\beta$
- *What it says:* The conditional mean of $Y$ is a linear function of $X$.
- *When violated:* The model is misspecified. $\hat{\beta}$ is biased and inconsistent -- it is fitting a line through a nonlinear cloud.
- *Detect:* Plot residuals vs. fitted values; look for systematic curvature. RESET test.
- *Fix:* Add polynomial terms, interactions, or use a nonlinear model.
2. No perfect multicollinearity: $\text{rank}(X) = p$
- *What it says:* No column of $X$ is an exact linear combination of the others. $(X^T X)^{-1}$ must exist.
- *When violated:* $(X^T X)$ is singular -- OLS has no unique solution. Near-multicollinearity inflates standard errors severely.
- *Detect:* Variance Inflation Factor (VIF). Rule of thumb: VIF $> 10$ is a concern.
- *Fix:* Drop a collinear variable, combine them (e.g., PCA), or use ridge regression.
3. Exogeneity: $E[\epsilon | X] = 0$
- *What it says:* The error is uncorrelated with the regressors on average. No omitted variable correlated with $X$, no reverse causality, no measurement error in $X$.
- *When violated:* $\hat{\beta}$ is biased and inconsistent. This is the most serious violation -- you cannot fix it by getting more data.
- *Detect:* Hausman test (compare OLS to IV). Qualitative reasoning about the data-generating process.
- *Fix:* Instrumental variables (IV/2SLS), control for omitted variables, or use a natural experiment.
4. Homoscedasticity: $\text{Var}(\epsilon_i | X) = \sigma^2$ (constant)
- *What it says:* All residuals have the same variance regardless of $X$.
- *When violated (heteroscedasticity):* $\hat{\beta}$ is still unbiased, but OLS is no longer efficient (GLS dominates). More importantly, standard errors are wrong, so t-stats and confidence intervals are invalid.
- *Detect:* Plot $|\hat{\epsilon}_i|$ vs. $\hat{y}_i$; Breusch-Pagan test; White test.
- *Fix:* Use White (heteroscedasticity-robust) standard errors, or WLS if the variance structure is known.
5. No autocorrelation: $\text{Cov}(\epsilon_i, \epsilon_j) = 0$ for $i \neq j$
- *What it says:* Residuals are uncorrelated across observations.
- *When violated:* Similar to heteroscedasticity -- $\hat{\beta}$ is unbiased but standard errors are wrong (usually understated), making inference anti-conservative.
- *Detect:* Durbin-Watson test; ACF plot of residuals.
- *Fix:* Newey-West (HAC) standard errors for time series; include lagged variables; GLS/FGLS.
Bonus -- Normality of $\epsilon$ (for finite-sample inference)
- *What it says:* $\epsilon \sim N(0, \sigma^2 I)$. Required for exact $t$ and $F$ distributions in small samples.
- *When violated:* Large-sample inference still works by CLT. Small-sample t-tests are only approximate.
- *Detect:* Q-Q plot; Jarque-Bera test.
- *Fix:* Increase sample size; bootstrap standard errors.
Answer: The Gauss-Markov assumptions are (1) linearity, (2) full rank, (3) exogeneity, (4) homoscedasticity, (5) no autocorrelation. Exogeneity violations bias $\hat{\beta}$; all others leave $\hat{\beta}$ unbiased but compromise efficiency or inference. Robust standard errors (White, Newey-West) are the workhorse fix for assumptions 4-5 in practice.
Intuition
The practical hierarchy for OLS violations is: endogeneity is catastrophic (biased estimates), heteroscedasticity and autocorrelation are annoying but fixable (use robust standard errors), and multicollinearity is mainly a nuisance (hard to separate individual effects, but the model's predictions are still fine). Most working quants deal with heteroscedasticity and autocorrelation routinely and reach for Newey-West or White SE as a default.
In financial time series, autocorrelation in residuals is almost guaranteed if you forget to include the right lagged variables or use overlapping returns. The Newey-West correction is standard in asset pricing papers for exactly this reason. Meanwhile, endogeneity is the hard problem -- if your alpha factor is correlated with something you did not control for, your backtest looks better than reality. Instrumental variables can fix it in principle, but good instruments are rare in finance, which is why experimental and quasi-experimental designs are increasingly valued.