Posterior for a Double-Headed Coin
A box contains 100 coins: 98 fair coins, 1 double-headed coin (heads on both sides), and 1 double-tailed coin (tails on both sides). You pick a coin uniformly at random and flip it.
- The first flip shows heads. Compute $P(\text{double-headed} \mid \text{heads})$, $P(\text{fair} \mid \text{heads})$, and $P(\text{double-tailed} \mid \text{heads})$.
- You flip the same coin again and it shows heads again. Update your posterior: compute $P(\text{double-headed} \mid \text{two heads})$.
Hints
- Set up three hypotheses (fair, double-headed, double-tailed) with priors given by the box composition. What is the likelihood of heads under each hypothesis?
- Apply Bayes' theorem: $P(\text{type} \mid H) = P(H \mid \text{type}) \cdot P(\text{type}) / P(H)$. Notice that $P(TT \mid H) = 0$ -- the double-tailed coin is ruled out immediately.
- For the second update, use the posterior from part (i) as the new prior. Each additional heads doubles the odds ratio of double-headed vs. fair.
Worked Solution
How to Think About It: This is a straightforward Bayes' theorem problem with three hypotheses: the coin is fair ($F$), double-headed ($HH$), or double-tailed ($TT$). The prior is determined by the composition of the box. After observing heads, the double-headed coin becomes more likely (it always shows heads), the fair coin stays plausible (50% chance of heads), and the double-tailed coin becomes impossible (it can never show heads). After a second heads, the double-headed coin becomes even more likely.
Quick Estimate: Before any flip, $P(HH) = 1/100$. The double-headed coin is 2x more likely to produce heads than a fair coin. After one heads, the posterior should roughly double the prior odds of $HH$ vs. $F$: from $1:98$ to about $2:98 = 1:49$, giving $P(HH \mid H) \approx 2/100 = 2\%$. After two heads, it doubles again: about $4:98$, so roughly $4\%$.
Approach: Apply Bayes' theorem sequentially.
Formal Solution:
Part (i): One flip shows heads.
Priors: $$P(HH) = \frac{1}{100}, \quad P(F) = \frac{98}{100}, \quad P(TT) = \frac{1}{100}$$
Likelihoods: $$P(H \mid HH) = 1, \quad P(H \mid F) = \frac{1}{2}, \quad P(H \mid TT) = 0$$
Total probability of heads: $$P(H) = 1 \cdot \frac{1}{100} + \frac{1}{2} \cdot \frac{98}{100} + 0 \cdot \frac{1}{100} = \frac{1}{100} + \frac{49}{100} = \frac{50}{100} = \frac{1}{2}$$
Posteriors after one heads: $$P(HH \mid H) = \frac{P(H \mid HH) \cdot P(HH)}{P(H)} = \frac{1 \cdot 1/100}{1/2} = \frac{2}{100} = \frac{1}{50}$$
$$P(F \mid H) = \frac{P(H \mid F) \cdot P(F)}{P(H)} = \frac{(1/2)(98/100)}{1/2} = \frac{98}{100} = \frac{49}{50}$$
$$P(TT \mid H) = \frac{P(H \mid TT) \cdot P(TT)}{P(H)} = \frac{0 \cdot 1/100}{1/2} = 0$$
Sanity check: $1/50 + 49/50 + 0 = 1$.
Part (ii): Second flip also shows heads (same coin).
Use the posteriors from part (i) as the new priors: $$P(HH \mid H) = \frac{1}{50}, \quad P(F \mid H) = \frac{49}{50}, \quad P(TT \mid H) = 0$$
Likelihoods for the second heads: $$P(H_2 \mid HH) = 1, \quad P(H_2 \mid F) = \frac{1}{2}, \quad P(H_2 \mid TT) = 0$$
Total probability of second heads given first heads: $$P(H_2 \mid H_1) = 1 \cdot \frac{1}{50} + \frac{1}{2} \cdot \frac{49}{50} + 0 = \frac{1}{50} + \frac{49}{100} = \frac{2}{100} + \frac{49}{100} = \frac{51}{100}$$
Posterior after two heads: $$P(HH \mid HH_{\text{obs}}) = \frac{1 \cdot 1/50}{51/100} = \frac{2/100}{51/100} = \frac{2}{51}$$
$$P(F \mid HH_{\text{obs}}) = \frac{(1/2)(49/50)}{51/100} = \frac{49/100}{51/100} = \frac{49}{51}$$
Sanity check: $2/51 + 49/51 = 1$.
Note the pattern: After $k$ consecutive heads, the posterior odds of $HH$ to $F$ are: $$\frac{P(HH \mid k \text{ heads})}{P(F \mid k \text{ heads})} = \frac{1}{98} \cdot 2^k$$
This is because each heads observation doubles the likelihood ratio of $HH$ vs. $F$.
Answer:
- After one heads: $P(HH \mid H) = 1/50 = 0.02$, $P(F \mid H) = 49/50 = 0.98$, $P(TT \mid H) = 0$.
- After two heads: $P(HH \mid \text{two heads}) = 2/51 \approx 0.0392$.
Intuition
This problem illustrates how Bayesian updating works as a likelihood-ratio machine. Each observation multiplies the prior odds by the likelihood ratio. For the double-headed coin vs. a fair coin, seeing heads has a likelihood ratio of $1 / (1/2) = 2$. So each heads doubles the odds in favor of the double-headed coin. Starting from prior odds $1:98$, after $k$ heads the odds become $2^k : 98$. You need about 7 heads ($2^7 = 128 > 98$) before the double-headed hypothesis becomes more likely than fair.
In trading, this same framework applies when you are trying to determine whether a signal is informative or just noise. Each correct prediction is like a "heads" -- it shifts your posterior toward "this signal is real." The question is always: how many confirming observations do you need before you act? The answer depends on your prior (how skeptical you are) and the likelihood ratio (how much more likely each observation is under the "real signal" hypothesis).