Mean and Variance of a Mixture of Normals
A random variable $X$ follows a two-component normal mixture: with probability $w$ it is drawn from $N(\mu_1, \sigma_1^2)$, and with probability $1-w$ it is drawn from $N(\mu_2, \sigma_2^2)$.
- Compute $E[X]$.
- Compute $\text{Var}(X)$. Your answer should make clear why a mixture can have higher variance than either component alone.
Hints
- Introduce a latent component indicator $C$ and use the tower property: $E[X] = E[E[X|C]]$.
- Apply the law of total variance: $\text{Var}(X) = E[\text{Var}(X|C)] + \text{Var}(E[X|C])$.
- For the second term, note that $E[X|C]$ is a discrete random variable taking value $\mu_1$ with probability $w$ and $\mu_2$ with probability $1-w$. Its variance is $w(1-w)(\mu_1-\mu_2)^2$.
Worked Solution
How to Think About It: Introduce a latent component indicator $C$ where $P(C=1) = w$ and $P(C=2) = 1-w$. Given $C$, $X$ is just a normal. The expectation is easy by linearity. The variance requires the law of total variance, which neatly splits into two interpretable pieces: the average within-component variance, and the variance of the component means (the between-component spread).
Quick Estimate: To see why the variance can be large, take $\mu_1 = -10$, $\mu_2 = +10$, $\sigma_1^2 = \sigma_2^2 = 1$, $w = 0.5$. Each component has variance $1$, but the mixture has mean $0$ and variance $\sigma^2_{\text{avg}} + w(1-w)(\Delta\mu)^2 = 1 + 0.25\cdot(20)^2 = 1 + 100 = 101$. The separation between the means (here $\Delta\mu = 20$, so the squared gap is $400$) contributes far more than the within-component spreads. This is the intuition behind fat tails in financial return distributions -- a low-volatility regime mixed with a high-volatility regime produces heavy tails.
Approach: Condition on the component indicator $C$. Part 1 uses the law of total expectation; Part 2 uses the law of total variance, $\text{Var}(X) = E[\text{Var}(X|C)] + \text{Var}(E[X|C])$, where the first term is the average within-component variance and the second is the spread of the component means.
Formal Solution:
Part 1: Expectation. By the law of total expectation:
$$E[X] = E[E[X|C]] = w \cdot E[X|C=1] + (1-w) \cdot E[X|C=2] = w\mu_1 + (1-w)\mu_2$$
Part 2: Variance. By the law of total variance:
$$\text{Var}(X) = E[\text{Var}(X|C)] + \text{Var}(E[X|C])$$
The first term is the expected within-component variance:
$$E[\text{Var}(X|C)] = w\sigma_1^2 + (1-w)\sigma_2^2$$
For the second term, $E[X|C]$ takes value $\mu_1$ with probability $w$ and $\mu_2$ with probability $1-w$. Its variance is:
$$\text{Var}(E[X|C]) = w\mu_1^2 + (1-w)\mu_2^2 - (w\mu_1 + (1-w)\mu_2)^2 = w(1-w)(\mu_1 - \mu_2)^2$$
Combining:
$$\text{Var}(X) = w\sigma_1^2 + (1-w)\sigma_2^2 + w(1-w)(\mu_1-\mu_2)^2$$
Answer:
$$E[X] = w\mu_1 + (1-w)\mu_2$$
$$\text{Var}(X) = w\sigma_1^2 + (1-w)\sigma_2^2 + w(1-w)(\mu_1-\mu_2)^2$$
The last term shows that even if both components have zero variance, the mixture has variance $w(1-w)(\mu_1-\mu_2)^2 > 0$ whenever the means differ.
Intuition
The law of total variance decomposition is one of the most useful tools in probability. It says: overall variance = average within-group variance + variance of group means. The second term is what makes mixtures interesting -- even if every component is tightly concentrated, if their means are far apart, the mixture has high variance. This is why mixture models are used to capture heavy-tailed distributions: a two-component mixture with different means (or different variances) can produce kurtosis far above that of a single Gaussian.
In finance, this structure appears constantly. Return distributions in volatility regime models are mixtures -- a low-vol and a high-vol regime. Even if each regime is Gaussian, the mixture is fat-tailed. The $w(1-w)(\mu_1-\mu_2)^2$ term is the between-regime variance, which is what blows up tail risk. Understanding this decomposition is the starting point for regime-switching models, and it is why simple Gaussian VaR models underestimate tail losses when regimes exist.