Expected Value of the Geometric PMF at Its Own Random Variable
Let $X \sim \text{Geom}(p)$ with PMF $p(k) = P(X = k) = p(1-p)^{k-1}$ for $k = 1, 2, 3, \ldots$
Compute $E[p(X)]$ -- the expected value of the PMF evaluated at the random variable itself -- when $p = 1/3$.
Note: There is a tempting but incorrect shortcut. Identify the error in the following reasoning, then compute the correct answer:
> "$E[p(X)] = E[P(X = X)] = E[1] = 1$."
Hints
- The notation $p(X)$ means: evaluate the PMF function $p(k) = p(1-p)^{k-1}$ at $k = X$. Do not try to write this as $P(X = X)$ -- that is a category error.
- Apply LOTUS: $E[p(X)] = \sum_{k=1}^{\infty} p(k) \cdot p(k) = \sum_{k=1}^{\infty} [p(k)]^2$. This is a sum of squared PMF values.
- After substituting the geometric PMF, you get $p^2 \sum_{k=1}^{\infty} [(1-p)^2]^{k-1}$, which is a geometric series with ratio $(1-p)^2$.
Worked Solution
How to Think About It: The expression $E[p(X)]$ is asking you to plug the random variable $X$ into its own PMF and then take the expectation. This is a perfectly well-defined quantity -- it is just $\sum_k p(k)^2$, a sum of squared PMF values. The wrong shortcut confuses two different uses of the variable: $P(X = X)$ is not a valid probability statement because you are conflating the random variable $X$ (which takes a specific value on each draw) with a generic argument $k$. You must evaluate $p(k)$ as a function of $k$ first, then substitute $X$ for $k$.
Quick Estimate: With $p = 1/3$, the PMF $p(k) = (1/3)(2/3)^{k-1}$ decays geometrically. The dominant term is $k=1$: $p(1) = 1/3$, contributing $(1/3)^2 = 1/9 \approx 0.111$ to $E[p(X)]$. The next term ($k=2$) contributes $(2/9)^2 = 4/81 \approx 0.049$. The sum will be dominated by small $k$, so a rough answer is around $0.11 + 0.05 + \ldots \approx 0.2$. We will see the exact answer is $1/5 = 0.2$.
Why the shortcut fails: The expression $P(X = X)$ is ambiguous. The correct reading of $p(X)$ is: fix the realization $X = k$, evaluate $p(k) = p(1-p)^{k-1}$, then take the expectation over $k$. Writing $P(X = X)$ treats the argument and the random variable as the same object in the probability statement, which is a category error.
Formal Solution: Apply LOTUS (Law of the Unconscious Statistician):
$$E[p(X)] = \sum_{k=1}^{\infty} p(k) \cdot p(k) = \sum_{k=1}^{\infty} [p(k)]^2 = \sum_{k=1}^{\infty} \left[p(1-p)^{k-1}\right]^2$$
$$= p^2 \sum_{k=1}^{\infty} (1-p)^{2(k-1)} = p^2 \sum_{j=0}^{\infty} \left[(1-p)^2\right]^j = \frac{p^2}{1-(1-p)^2}$$
Simplify the denominator: $1-(1-p)^2 = 1 - 1 + 2p - p^2 = 2p - p^2 = p(2-p)$. So:
$$E[p(X)] = \frac{p^2}{p(2-p)} = \frac{p}{2-p}$$
Sanity check: As $p \to 1$, $X = 1$ with probability $1$, so $p(X) = p(1) = 1$ with probability $1$, giving $E[p(X)] = 1$. Formula: $\frac{1}{2-1} = 1$. Correct. As $p \to 0$, the PMF spreads out and $E[p(X)] \to 0$. Formula: $\frac{p}{2} \to 0$. Correct.
Answer: For $p = 1/3$:
$$E[p(X)] = \frac{1/3}{2 - 1/3} = \frac{1/3}{5/3} = \frac{1}{5}$$
Intuition
This problem is a nice trap. The shortcut $E[P(X=X)] = 1$ looks superficially reasonable because $P(X = x) = 1$ if you fix $x$ and compare it to itself -- but that conflates the random variable and its argument. The correct interpretation of $E[p(X)]$ is straightforward once you see it: you are computing the expected squared PMF value, $\sum_k [p(k)]^2$, which is sometimes called the collision probability or self-information measure. It measures how concentrated the distribution is -- a distribution that puts all mass at one point has $E[p(X)] = 1$, while a very spread-out distribution has $E[p(X)] \approx 0$.
LOTUS is the right tool whenever you need $E[g(X)]$ for some function $g$ of a random variable -- just sum (or integrate) $g(k) \cdot P(X=k)$ over all $k$. The mistake of writing $P(X=X)$ is a classic trick question in probability courses precisely because it exploits the overloaded notation of the PMF.