Bayesian Coin Identification
You have two coins: Coin A is fair ($p = 0.5$) and Coin B is biased ($p = 0.7$). You pick one uniformly at random and flip it 5 times, observing 4 heads and 1 tail. What is the posterior probability that you are holding Coin B?
Hints
- With only two hypotheses and a uniform prior, Bayes' theorem reduces to: posterior odds = likelihood ratio. Compute $P(D|B) / P(D|A)$ first.
- The likelihood of observing 4 heads in 5 flips from a coin with bias $p$ is $\binom{5}{4} p^4 (1-p)$. The binomial coefficient $\binom{5}{4} = 5$ appears in both likelihoods and cancels in the odds ratio.
- Posterior odds for B over A = prior odds $\times$ likelihood ratio = $1 \times P(D|B)/P(D|A)$. Convert to probability: $P(B|D) = \text{odds} / (1 + \text{odds})$.
Worked Solution
How to Think About It: Before flipping, you have a 50-50 chance of holding either coin. The data -- 4 heads in 5 flips -- is more likely under Coin B ($p = 0.7$) than Coin A ($p = 0.5$), so the posterior should shift toward B. The question is by how much. Intuitively, 4/5 heads (80%) is much closer to 70% than to 50%, so the update should be substantial but not overwhelming -- you should end up somewhere around 65-75% probability for Coin B.
Quick Estimate: The likelihood ratio is $P(D | B) / P(D | A) = (0.7^4 \times 0.3) / (0.5^4 \times 0.5)$. Numerically, $0.7^4 \times 0.3 \approx 0.2401 \times 0.3 = 0.0720$ and $0.5^4 \times 0.5 = 0.5^5 = 0.03125$, so the likelihood ratio $\approx 0.0720 / 0.03125 \approx 2.3$. Starting from 50-50 prior odds, posterior odds for B are $2.3:1$, so $P(B|D) \approx 2.3/3.3 \approx 0.70$.
Approach: Apply Bayes' theorem with binomial likelihoods.
Formal Solution:
Let $D$ denote the observed data (4 heads, 1 tail in 5 flips).
Prior: $P(A) = P(B) = 1/2$.
Likelihoods (the $\binom{5}{4}$ factor cancels in the ratio, but include for clarity): $$P(D | A) = \binom{5}{4}(0.5)^4(0.5)^1 = 5 \times \frac{1}{32} = \frac{5}{32} \approx 0.15625$$ $$P(D | B) = \binom{5}{4}(0.7)^4(0.3)^1 = 5 \times 0.2401 \times 0.3 \approx 0.36015$$
Marginal likelihood: $$P(D) = P(D|A) \cdot P(A) + P(D|B) \cdot P(B) = \frac{1}{2}(0.15625 + 0.36015) = \frac{0.5164}{2} \approx 0.25820$$
Posterior: $$P(B|D) = \frac{P(D|B) \cdot P(B)}{P(D)} = \frac{0.36015 \times 0.5}{0.25820} \approx \frac{0.18008}{0.25820} \approx \boxed{0.6975}$$
The posterior shifts from 50% to about 69.8% -- consistent with our quick estimate of ~70%.
Answer: $P(B | \text{4H, 1T}) \approx 0.698$. The biased coin is now roughly 2.3 times more likely than the fair coin.
Intuition
Bayes' theorem is the correct machinery for updating beliefs in light of evidence, and this problem shows how it works in a concrete setting. The key quantity is the likelihood ratio -- how much more (or less) probable the observed data is under each hypothesis. Here, 4 heads in 5 flips is about 2.3 times more likely under the biased coin than the fair coin, so the posterior odds for B shift by exactly that factor. This is why a practitioner thinks in likelihood ratios: they directly translate to posterior odds when the prior is symmetric.
A common mistake is to look at 80% heads and jump to 'obviously Coin B' -- but with only 5 flips, you simply do not have enough data to be nearly certain. The posterior of ~70% reflects genuine remaining uncertainty. If you observed 40 heads in 50 flips, the likelihood ratio would be $(0.7/0.5)^{40} \times (0.3/0.5)^{10} \approx 2.3^{40} \times 0.6^{10}$, which is astronomically large, and the posterior for B would be essentially 1. More data collapses the uncertainty -- that is Bayesian learning.