Feasible Range of Third Pairwise Correlation

Worked Solution

How to Think About It: Whenever you see "what values can a correlation take given other correlations," the tool is the positive semidefinite (PSD) constraint on the correlation matrix. A correlation matrix is PSD because $w^{T}Rw = \text{Var}(w_1\tilde X + w_2\tilde Y + w_3\tilde Z) \geq 0$ for the standardized variables and any weights $w$. A valid correlation matrix must have all eigenvalues $\geq 0$, which is equivalent to requiring all principal minors (determinants of submatrices) to be nonnegative. For a $3 \times 3$ matrix, the binding constraint is $\det(R) \geq 0$. Before computing, think about limiting cases: if $\rho = 1$, then $X = Y = Z$ (up to affine transforms), so $r$ must equal 1. If $\rho = 0$, then $X$ and $Z$ are each uncorrelated with $Y$ but could be anything with respect to each other, so $r \in [-1, 1]$. We should expect the feasible range to shrink as $|\rho|$ grows.

Quick Estimate: Take $\rho = 0.5$. The third correlation $r$ is certainly at most 1. For the lower bound, guess: $X$ and $Z$ each have correlation $0.5$ with $Y$, so they share "half" their variation with $Y$. How anti-correlated can $X$ and $Z$ be? Split each of $X$ and $Z$ into a part along $Y$ (correlation contribution $\rho \cdot \rho = 0.25$) and a residual of variance $1 - \rho^2 = 0.75$. At worst the two residuals are perfectly negatively correlated. If those residuals are perfectly negatively correlated, $r = \rho^2 + (-1)(1 - \rho^2) = 2\rho^2 - 1 = -0.5$. So guess $r \in [-0.5, 1]$.

Approach: Write down the $3 \times 3$ correlation matrix and enforce $\det(R) \geq 0$, which gives a quadratic inequality in $r$.

Formal Solution:

The correlation matrix is:

$$R = \begin{pmatrix} 1 & \rho & r \\ \rho & 1 & \rho \\ r & \rho & 1 \end{pmatrix}$$

Compute the determinant by cofactor expansion along the first row:

$$\det(R) = 1 \cdot (1 - \rho^2) - \rho \cdot (\rho - r\rho) + r \cdot (\rho^2 - r)$$

• First term: $1 \cdot (1 \cdot 1 - \rho \cdot \rho) = 1 - \rho^2$
• Second term: $-\rho \cdot (\rho \cdot 1 - \rho \cdot r) = -\rho(\rho - r\rho) = -\rho^2 + r\rho^2$
• Third term: $r \cdot (\rho \cdot \rho - 1 \cdot r) = r(\rho^2 - r) = r\rho^2 - r^2$

Summing these:

$$\det(R) = 1 - \rho^2 - \rho^2 + r\rho^2 + r\rho^2 - r^2 = 1 - 2\rho^2 + 2r\rho^2 - r^2$$

Setting $\det(R) \geq 0$:

$$1 - 2\rho^2 + 2r\rho^2 - r^2 \geq 0$$

Rearrange as a quadratic in $r$:

$$r^2 - 2\rho^2 r + (2\rho^2 - 1) \leq 0$$

Apply the quadratic formula to find the roots:

$$r = \frac{2\rho^2 \pm \sqrt{4\rho^4 - 4(2\rho^2 - 1)}}{2} = \rho^2 \pm \sqrt{\rho^4 - 2\rho^2 + 1}$$

The discriminant simplifies:

$$\rho^4 - 2\rho^2 + 1 = (\rho^2 - 1)^2 = (1 - \rho^2)^2$$

So:

$$r = \rho^2 \pm (1 - \rho^2)$$

This gives the two roots:

• Upper root: $\rho^2 + (1 - \rho^2) = 1$
• Lower root: $\rho^2 - (1 - \rho^2) = 2\rho^2 - 1$

Since the quadratic in $r$ opens upward (coefficient of $r^2$ is positive), the inequality $\leq 0$ holds between the roots:

$$r \in [2\rho^2 - 1, \; 1]$$

Note that $2\rho^2 - 1 \geq -1$ for all $\rho$, so this is always a valid sub-interval of $[-1, 1]$.

Verification: For $\rho = 0.5$: $r \in [2(0.25) - 1, 1] = [-0.5, 1]$. For $\rho = 1$: $r \in [1, 1]$, so $r = 1$ is forced. For $\rho = 0$: $r \in [-1, 1]$, meaning no constraint at all. For $\rho = 0.8$: $r \in [0.28, 1]$, so two variables each correlated $0.8$ with $Y$ must be positively correlated with each other. All match our intuition. (The $2 \times 2$ principal minors $1 - \rho^2$ and $1 - r^2$ are automatically nonnegative on this interval, so $\det R \geq 0$ is the only binding condition.)

Geometric interpretation and a concrete example: Write $X = \rho Y + \sqrt{1 - \rho^2} \, U$ and $Z = \rho Y + \sqrt{1 - \rho^2} \, V$ where $U, V$ are unit-variance and uncorrelated with $Y$. Then $\text{Corr}(X, Z) = \rho^2 + (1 - \rho^2) \text{Corr}(U, V)$. Since $\text{Corr}(U, V) \in [-1, 1]$, we get the same bounds, and every value in between is attained. Concretely, take $Y, U$ independent standard normals and $V = -U$: then $X = \rho Y + \sqrt{1-\rho^2}\,U$ and $Z = \rho Y - \sqrt{1-\rho^2}\,U$ have unit variance, correlation $\rho$ with $Y$, and $\text{Corr}(X, Z) = \rho^2 - (1 - \rho^2) = 2\rho^2 - 1$. Taking $Z = X$ attains $1$. (A simulation with $\rho = 0.5$ gives sample correlations $0.50$, $0.50$, $-0.50$.) In angle terms, $\rho = \cos\theta$ puts $X$ and $Z$ on a cone of half-angle $\theta$ around $Y$, so they are between $0$ and $2\theta$ apart, and $\cos 2\theta = 2\rho^2 - 1$.

Part (b): Unequal correlations. With $\text{Corr}(X, Y) = \alpha$ and $\text{Corr}(Y, Z) = \beta$, the same expansion gives $$\det R = 1 - \alpha^2 - \beta^2 - r^2 + 2\alpha\beta r \ge 0 \iff (r - \alpha\beta)^2 \le (1 - \alpha^2)(1 - \beta^2),$$ so $$r \in \left[\alpha\beta - \sqrt{(1-\alpha^2)(1-\beta^2)},\ \alpha\beta + \sqrt{(1-\alpha^2)(1-\beta^2)}\right].$$ (The same decomposition shows it: $\text{Corr}(X, Z) = \alpha\beta + \sqrt{(1-\alpha^2)(1-\beta^2)}\,\text{Corr}(U, V)$.) With $\alpha = \beta = \rho$ this is $\rho^2 \pm (1 - \rho^2)$, part (a) again.

For $\alpha = 0.8$, $\beta = 0.9$: $\alpha\beta = 0.72$ and $\sqrt{0.36 \cdot 0.19} = 0.6\sqrt{0.19} \approx 0.2615$, so $$\text{Corr}(X, Z) \in [0.4585,\ 0.9815].$$ Angles confirm it: $\arccos 0.8 \approx 36.87^\circ$ and $\arccos 0.9 \approx 25.84^\circ$, so the angle between $X$ and $Z$ lies between their difference $11.03^\circ$ and their sum $62.71^\circ$, whose cosines are $0.9815$ and $0.4585$. Two variables both tied tightly to $Y$ cannot even be uncorrelated with each other, and they cannot be perfectly correlated either, because they hug $Y$ at different strengths. The smallest eigenvalue of $R$ is zero at both endpoints and turns negative just outside them.

Answer: (a) $\text{Corr}(X, Z) \in [2\rho^2 - 1, \; 1]$ (for example $[-0.5, 1]$ at $\rho = 0.5$ and $[0.28, 1]$ at $\rho = 0.8$). (b) In general $\text{Corr}(X, Z) \in \alpha\beta \pm \sqrt{(1-\alpha^2)(1-\beta^2)}$; for $0.8$ and $0.9$ this is $[0.4585, 0.9815]$.