OLS with Correlated Errors
Consider the standard linear regression model $y = X\beta + \varepsilon$, where OLS assumes $\text{Cov}(\varepsilon) = \sigma^2 I$. Now suppose the errors are not i.i.d. but instead exhibit correlation.
- If the errors are positively correlated, what happens to the OLS coefficient estimates $\hat{\beta}$? Are they still unbiased? What about the estimated standard errors and the resulting t-statistics?
- If the errors are negatively correlated, how does the story change?
- What would you do in practice to fix these issues?
Hints
- Unbiasedness of $\hat{\beta}$ depends on $E[\varepsilon | X] = 0$, not on the covariance structure of $\varepsilon$. Separate the point estimate from the uncertainty estimate.
- The true variance of $\hat{\beta}$ is $(X^T X)^{-1} X^T \Omega X (X^T X)^{-1}$, which differs from the OLS formula $(X^T X)^{-1} \sigma^2$ when $\Omega \neq \sigma^2 I$.
- Positive correlation makes the off-diagonal entries of $\Omega$ positive, inflating the true variance above what OLS reports. Negative correlation does the opposite. Think about what this does to your t-statistics.
Worked Solution
How to Think About It: The key insight is separating two things: the OLS point estimates themselves versus the uncertainty estimates around them. OLS computes $\hat{\beta} = (X^T X)^{-1} X^T y$. That formula does not depend on the error covariance structure at all -- as long as $E[\varepsilon | X] = 0$, the estimator is unbiased regardless of correlation. The problem is with the standard errors. OLS computes standard errors assuming $\text{Cov}(\varepsilon) = \sigma^2 I$, and when that is wrong, the standard errors are wrong, which means your t-stats and p-values are garbage. The direction of the bias depends on whether correlations are positive or negative.
Key Insight: OLS coefficients stay unbiased under correlated errors; it is the inference (standard errors, hypothesis tests, confidence intervals) that breaks.
The Method:
Case 1: Positively correlated errors
- $\hat{\beta}$ remains unbiased. Unbiasedness only requires $E[\varepsilon | X] = 0$, which has nothing to do with the covariance structure.
- $\hat{\beta}$ is inefficient -- it is no longer the minimum-variance linear unbiased estimator (the Gauss-Markov theorem fails when $\text{Cov}(\varepsilon) \neq \sigma^2 I$).
- The true variance of $\hat{\beta}$ is $(X^T X)^{-1} X^T \Omega X (X^T X)^{-1}$, where $\Omega = \text{Cov}(\varepsilon)$. When errors are positively correlated, the off-diagonal terms of $\Omega$ are positive, and the true variance of $\hat{\beta}$ is typically larger than what OLS reports.
- Consequence: OLS underestimates standard errors. Your t-statistics are inflated and p-values are too small, leading to spurious significance. You think you have found a signal when you have not.
Case 2: Negatively correlated errors
- $\hat{\beta}$ is still unbiased for the same reason.
- Still inefficient.
- With negative correlation, the off-diagonal terms of $\Omega$ are negative. The true variance of $\hat{\beta}$ is typically smaller than what OLS reports.
- Consequence: OLS overestimates standard errors. Your t-statistics are deflated and p-values are too large. You may fail to detect real effects -- a loss of power.
Practical Considerations:
- GLS (Generalized Least Squares): If you know $\Omega$, use $\hat{\beta}_{GLS} = (X^T \Omega^{-1} X)^{-1} X^T \Omega^{-1} y$. This is BLUE (best linear unbiased estimator) under the true covariance structure.
- Feasible GLS (FGLS): In practice you estimate $\Omega$ from the residuals and plug it in. Works well with large samples.
- HAC Standard Errors: If you just want correct inference without changing $\hat{\beta}$, use Newey-West (for time series) or clustered standard errors (for panel data). These give you the right standard errors without specifying the full covariance structure.
- Rule of thumb in finance: Positive serial correlation in residuals is far more common (think trending macro variables, autocorrelated returns at low frequency). Always check for it with a Durbin-Watson test or by plotting residual autocorrelations.
Answer: OLS coefficient estimates remain unbiased regardless of error correlation. With positive correlation, OLS underestimates standard errors (inflated t-stats, false significance). With negative correlation, OLS overestimates standard errors (deflated t-stats, missed effects). Fix: use GLS or HAC/Newey-West robust standard errors.
Intuition
The core lesson here is that OLS is remarkably robust for point estimation but fragile for inference. The formula $\hat{\beta} = (X^T X)^{-1} X^T y$ does not "know" about the error covariance -- it just projects. So the estimates stay unbiased. But the standard errors are computed under a specific assumption about $\text{Cov}(\varepsilon)$, and when that assumption is wrong, the standard errors are wrong in a predictable direction.
In finance, this matters constantly. Time series residuals are almost always positively autocorrelated at some lag, which means naive OLS standard errors are too small and you will find "significant" predictors that are actually noise. This is one of the main reasons quant researchers use Newey-West standard errors as a default -- it is cheap insurance against correlated errors destroying your inference.