Order Statistics: Expectations and Densities for Uniform Samples
Let $X_1, X_2, \ldots, X_n$ be $n$ i.i.d. uniform random variables on $[0, 1]$. Denote their order statistics $X_{(1)} \leq X_{(2)} \leq \cdots \leq X_{(n)}$.
(a) Prove that $E[X_{(k)}] = \dfrac{k}{n+1}$ for any $k \in \{1, 2, \ldots, n\}$.
(b) Derive the probability density function of $X_{(k)}$ directly -- without first finding the CDF -- and identify the resulting distribution by name and parameters.
Hints
- For part (a), think geometrically: $n$ uniform points on $[0,1]$ create $n+1$ gaps. By symmetry, all gaps have equal expected length. The $k$-th order statistic is the sum of the first $k$ gaps.
- For part (b), write out what $P(X_{(k)} \in [x, x+dx])$ means combinatorially: exactly $k-1$ points below $x$, one in the interval, $n-k$ above. The multinomial coefficient counts the arrangements.
- The PDF you derive will be proportional to $x^{k-1}(1-x)^{n-k}$. Recognize this as a Beta distribution: Beta$(a, b)$ has density $\propto x^{a-1}(1-x)^{b-1}$, so match $a = k$ and $b = n - k + 1$.
Worked Solution
How to Think About It: Drop $n$ points uniformly on $[0,1]$. By symmetry they create $n + 1$ gaps of equal expected length $1/(n+1)$ each. The $k$-th order statistic is the right endpoint of the $k$-th gap, so its expected value is $k/(n+1)$. That is the intuition for part (a) -- no calculation required. For part (b), the direct approach to the PDF uses a counting argument: how many ways can you assign the $n$ points so that the $k$-th smallest lands in a small interval $[x, x + dx]$?
(a) Proof of $E[X_{(k)}] = k/(n+1)$:
Symmetry/gap argument: The $n$ points partition $[0,1]$ into $n+1$ gaps. By symmetry of the uniform distribution, all $n+1$ gap lengths are exchangeable (identically distributed). Their expected values are therefore equal, and since they sum to 1: $$E[\text{gap length}] = \frac{1}{n+1}$$ The $k$-th order statistic equals the sum of the first $k$ gaps, so: $$E[X_{(k)}] = k \cdot \frac{1}{n+1} = \frac{k}{n+1} \quad \square$$
Formal verification via distribution: As shown in part (b), $X_{(k)} \sim \text{Beta}(k, n-k+1)$. The mean of a Beta$(a,b)$ is $a/(a+b)$, giving: $$E[X_{(k)}] = \frac{k}{k + (n-k+1)} = \frac{k}{n+1} \quad \square$$
(b) PDF of $X_{(k)}$ without using the CDF:
Consider the event $X_{(k)} \in [x, x+dx]$ for small $dx > 0$ and $x \in (0,1)$.
For the $k$-th order statistic to land in $[x, x + dx]$, we need exactly: - $k - 1$ of the $n$ points fall in $[0, x)$ - exactly $1$ point falls in $[x, x + dx)$ - $n - k$ points fall in $(x + dx, 1]$
The multinomial coefficient for choosing which points go where is $\dbinom{n}{k-1, 1, n-k} = \dfrac{n!}{(k-1)! \cdot 1! \cdot (n-k)!}$.
The probabilities are: $(x)^{k-1}$ for the $k-1$ points below $x$, $(dx)$ for the 1 point in the interval, and $(1-x)^{n-k}$ for the $n-k$ points above. Therefore: $$P(X_{(k)} \in [x, x+dx]) = \frac{n!}{(k-1)!(n-k)!} x^{k-1}(1-x)^{n-k} \, dx$$
This gives the PDF: $$f_{X_{(k)}}(x) = \frac{n!}{(k-1)!(n-k)!} x^{k-1}(1-x)^{n-k}, \quad x \in [0,1]$$
This is exactly the Beta$(k, n-k+1)$ density, since $B(k, n-k+1) = \frac{(k-1)!(n-k)!}{n!}$: $$f_{X_{(k)}}(x) = \frac{x^{k-1}(1-x)^{(n-k+1)-1}}{B(k, n-k+1)}, \quad x \in [0,1]$$
Answer: (a) $E[X_{(k)}] = k/(n+1)$ by the symmetry of the $n+1$ uniform gaps. (b) $X_{(k)} \sim \text{Beta}(k, n-k+1)$ with PDF $f(x) = \frac{n!}{(k-1)!(n-k)!} x^{k-1}(1-x)^{n-k}$.
Intuition
The connection between order statistics of the uniform distribution and the Beta distribution is one of the most useful facts in probability. It says that if you want to model a "middle value" or a "top quantile" from a sample, Beta distributions are your natural language. And the mean $k/(n+1)$ is beautifully interpretable: it is just the fraction of the way through $[0,1]$ you would expect the $k$-th-ranked point to sit.
This result has direct applications in quantitative finance. The order statistics of $n$ i.i.d. uniform random variables are the building blocks for empirical CDFs and quantile estimation. In risk management, the $k$-th smallest loss from $n$ scenarios is approximately Beta-distributed, which lets you put confidence intervals on VaR estimates. The gap argument also appears in analysis of trading auctions and bid-ask ladder models, where you care about where the next trade price lands relative to a set of resting orders.