Joint Density and Spread of Uniform Order Statistics

Worked Solution

How to Think About It: Order statistics of uniforms are the bread and butter of quantitative probability. The key mental picture: if you drop $n$ points uniformly on $[0,1]$ and sort them, the $k$-th smallest lands near $k/(n+1)$ on average, and the gaps between them behave like rescaled exponentials. This is exactly the connection to Poisson processes -- uniform order statistics are Poisson arrival times in disguise (after a time change). Before diving into densities, anchor yourself: $E[U_{(k)}] = k/(n+1)$, so the expected spread $E[U_{(j)} - U_{(i)}]$ should be $(j - i)/(n+1)$. That is your quick sanity check.

Quick Estimate: For a concrete case, take $n = 10$, $i = 3$, $j = 7$. The expected spread is $(7 - 3)/(10 + 1) = 4/11 \approx 0.364$. For the variance, we will derive that $\text{Var}(U_{(k)}) = k(n + 1 - k)/((n+1)^2(n+2))$. With $n = 10$, $\text{Var}(U_{(3)}) = 3 \cdot 8 / (121 \cdot 12) = 24/1452 \approx 0.0165$ and $\text{Var}(U_{(7)}) = 7 \cdot 4 / 1452 = 28/1452 \approx 0.0193$. The covariance $\text{Cov}(U_{(i)}, U_{(j)}) = i(n+1-j)/((n+1)^2(n+2))$ gives $3 \cdot 4/1452 = 12/1452 \approx 0.00826$. So $\text{Var}(U_{(7)} - U_{(3)}) \approx 0.0165 + 0.0193 - 2(0.00826) \approx 0.0193$. Let us now verify this with the general formula.

Approach: We derive the joint density by a multinomial counting argument, then read off moments from the known Beta marginals and use the Poisson-uniform connection.

Part 1: Joint Density of $(U_{(i)}, U_{(j)})$

Partition $[0,1]$ into regions determined by values $x < y$: - $i - 1$ variables fall in $[0, x)$, - 1 variable equals $x$ (the $i$-th order statistic), - $j - i - 1$ variables fall in $(x, y)$, - 1 variable equals $y$ (the $j$-th order statistic), - $n - j$ variables fall in $(y, 1]$.

The multinomial coefficient for this arrangement is:

$\frac{n!}{(i-1)! \cdot 1! \cdot (j-i-1)! \cdot 1! \cdot (n-j)!}$

Since each $U_k \sim \text{Uniform}(0,1)$, the probability element for each region contributes the length raised to the count. The joint density is:

$f_{U_{(i)}, U_{(j)}}(x, y) = \frac{n!}{(i-1)!(j-i-1)!(n-j)!} \, x^{i-1}(y - x)^{j-i-1}(1 - y)^{n-j}$

for $0 < x < y < 1$.

Part 2: $E[U_{(j)} - U_{(i)}]$ and $\text{Var}(U_{(j)} - U_{(i)})]$

The marginal density of $U_{(k)}$ is $\text{Beta}(k, n+1-k)$, giving:

$E[U_{(k)}] = \frac{k}{n+1}, \quad \text{Var}(U_{(k)}) = \frac{k(n+1-k)}{(n+1)^2(n+2)}$

By linearity:

$E[U_{(j)} - U_{(i)}] = \frac{j}{n+1} - \frac{i}{n+1} = \frac{j - i}{n+1}$

For the variance, we need $\text{Cov}(U_{(i)}, U_{(j)})$. A clean derivation: for $i \leq j$,

$E[U_{(i)} U_{(j)}] = \int_0^1 \int_0^y x \cdot y \cdot f_{U_{(i)}, U_{(j)}}(x, y) \, dx \, dy$

Using the Beta integral identity repeatedly, this evaluates to:

$E[U_{(i)} U_{(j)}] = \frac{i(j+1)}{(n+1)(n+2)}$

So:

$\text{Cov}(U_{(i)}, U_{(j)}) = \frac{i(j+1)}{(n+1)(n+2)} - \frac{ij}{(n+1)^2} = \frac{i(n+1-j)}{(n+1)^2(n+2)}$

Note the elegant form: the covariance of $U_{(i)}$ and $U_{(j)}$ equals $\text{Var}(U_{(i)})$ scaled by $(n+1-j)/(n+1-i)$. It is always positive (knowing the $i$-th order statistic is large tells you the $j$-th is likely large too). Now:

$\text{Var}(U_{(j)} - U_{(i)}) = \text{Var}(U_{(j)}) + \text{Var}(U_{(i)}) - 2\text{Cov}(U_{(i)}, U_{(j)})$

$= \frac{j(n+1-j)}{(n+1)^2(n+2)} + \frac{i(n+1-i)}{(n+1)^2(n+2)} - \frac{2i(n+1-j)}{(n+1)^2(n+2)}$

$= \frac{j(n+1-j) + i(n+1-i) - 2i(n+1-j)}{(n+1)^2(n+2)}$

Expanding the numerator:

$j(n+1) - j^2 + i(n+1) - i^2 - 2i(n+1) + 2ij$ $= (j - i)(n+1) - (j^2 - 2ij + i^2) = (j-i)(n+1) - (j-i)^2 = (j-i)(n+1-j+i)$

Therefore:

$\boxed{\text{Var}(U_{(j)} - U_{(i)}) = \frac{(j - i)(n + 1 - j + i)}{(n+1)^2(n+2)}}$

Sanity check with our earlier example: $n = 10$, $i = 3$, $j = 7$: numerator is $(j-i)(n+1-j+i) = 4 \cdot 7 = 28$, denominator is