OLS with Correlated Errors

Worked Solution

How to Think About It: The key insight is separating two things: the OLS point estimates themselves versus the uncertainty estimates around them. OLS computes $\hat{\beta} = (X^T X)^{-1} X^T y$. That formula does not depend on the error covariance structure at all -- as long as $E[\varepsilon | X] = 0$, the estimator is unbiased regardless of correlation. The problem is with the standard errors. OLS computes standard errors assuming $\text{Cov}(\varepsilon) = \sigma^2 I$, and when that is wrong, the standard errors are wrong, which means your t-stats and p-values are garbage. The direction in which they are wrong depends on how the error covariance lines up with the regressors, not on the sign of the correlation alone.

Key Insight: OLS coefficients stay unbiased under correlated errors; it is the inference (standard errors, hypothesis tests, confidence intervals) that breaks.

The Method:

Case 1: Positively correlated errors

• $\hat{\beta}$ remains unbiased. Unbiasedness only requires $E[\varepsilon | X] = 0$, which has nothing to do with the covariance structure.
• $\hat{\beta}$ is inefficient -- it is no longer the minimum-variance linear unbiased estimator (the Gauss-Markov theorem fails when $\text{Cov}(\varepsilon) \neq \sigma^2 I$).
• The true variance of $\hat{\beta}$ is $(X^T X)^{-1} X^T \Omega X (X^T X)^{-1}$, where $\Omega = \text{Cov}(\varepsilon)$. When errors are positively correlated, the off-diagonal terms of $\Omega$ are positive, and in the canonical time-series case (a persistent regressor) the true variance of $\hat{\beta}$ comes out larger than what OLS reports. That direction is not settled by the sign of the correlation alone -- it depends on how $\Omega$ lines up with $X$. A contrast regressor $X = (1, -1)^T$ reverses it: with two positively correlated errors the true variance is $\sigma^2(1 - \rho)/2$, below the OLS value $\sigma^2/2$.
• Consequence in that typical case: OLS underestimates standard errors. Your t-statistics are inflated and p-values are too small, leading to spurious significance. You think you have found a signal when you have not.

Case 2: Negatively correlated errors

• $\hat{\beta}$ is still unbiased for the same reason.
• Still inefficient.
• With negative correlation, the off-diagonal terms of $\Omega$ are negative, and with a persistent regressor the true variance of $\hat{\beta}$ comes out smaller than what OLS reports. This too is the typical case rather than a theorem -- the same dependence on how $\Omega$ lines up with $X$ applies.
• Consequence in that typical case: OLS overestimates standard errors. Your t-statistics are deflated and p-values are too large. You may fail to detect real effects -- a loss of power.

Practical Considerations:

1. GLS (Generalized Least Squares): If you know $\Omega$, use $\hat{\beta}_{GLS} = (X^T \Omega^{-1} X)^{-1} X^T \Omega^{-1} y$. This is BLUE (best linear unbiased estimator) under the true covariance structure.
2. Feasible GLS (FGLS): In practice you estimate $\Omega$ from the residuals and plug it in. Works well with large samples.
3. HAC Standard Errors: If you just want correct inference without changing $\hat{\beta}$, use Newey-West (for time series) or clustered standard errors (for panel data). These give you the right standard errors without specifying the full covariance structure.
4. Rule of thumb in finance: Positive serial correlation in residuals is far more common (think trending macro variables, autocorrelated returns at low frequency). Always check for it with a Durbin-Watson test or by plotting residual autocorrelations.

Answer: OLS coefficient estimates remain unbiased regardless of error correlation, but the OLS standard errors are no longer valid: the true variance is the sandwich $(X^T X)^{-1} X^T \Omega X (X^T X)^{-1}$, not $\sigma^2 (X^T X)^{-1}$. Which way they err depends on how $\Omega$ lines up with $X$, not on the sign of the correlation alone. In the canonical time-series case (positively autocorrelated errors, persistent regressor) OLS understates the standard errors and inflates t-stats; with negatively correlated errors it typically overstates them and costs power. Fix: use GLS or HAC/Newey-West robust standard errors.