Bayesian Coin Inference and Kelly Betting with Beta Prior

Worked Solution

How to Think About It: This is the canonical Bayesian conjugate setup -- Beta prior meets Binomial likelihood and produces a Beta posterior. The posterior predictive for future flips is a Beta-Binomial, which accounts for your remaining uncertainty about $p$. The Kelly part is a direct application: given your posterior belief about the coin's bias, Kelly tells you to bet on the more likely side with a specific fraction.

Quick Estimate: Suppose $a = b = 1$ (uniform prior), $H = 7$, $T = 3$. Posterior is $\text{Beta}(8, 4)$, posterior mean $\hat{p} = 8/12 = 2/3$. Kelly fraction for an even-money bet on heads: $f^{*} = 2\hat{p} - 1 = 1/3$, so bet one-third of your bankroll on heads.

Formal Solution:

Part (i): Posterior distribution

The Beta-Binomial conjugacy gives: $$p \mid \text{data} \sim \text{Beta}(a + H, \, b + T)$$

Let $a' = a + H$ and $b' = b + T$ for convenience. The posterior mean is: $$E[p \mid \text{data}] = \frac{a'}{a' + b'}$$

The posterior predictive probability that the next flip is heads: $$P(\text{next flip} = H \mid \text{data}) = E[p \mid \text{data}] = \frac{a + H}{a + b + H + T}$$

This is just the posterior mean -- a standard result from exchangeability.

Part (ii): Posterior predictive for $k$ flips

The posterior predictive distribution for the number of heads $h$ in the next $k$ flips is Beta-Binomial: $$P(h \mid k, a', b') = \binom{k}{h} \frac{B(a' + h, \, b' + k - h)}{B(a', b')}$$

where $B(\cdot, \cdot)$ is the Beta function. This can be written using rising factorials: $$P(h \mid k, a', b') = \binom{k}{h} \frac{\prod_{i=0}^{h-1}(a' + i) \cdot \prod_{j=0}^{k-h-1}(b' + j)}{\prod_{m=0}^{k-1}(a' + b' + m)}$$

The mean of this distribution is $k \cdot a'/(a' + b')$ and the variance is larger than the Binomial variance because of the extra uncertainty in $p$ (over-dispersion).

Part (iii): Kelly criterion

Let $q = P(\text{next flip} = H \mid \text{data}) = a'/(a' + b')$ be your posterior probability of heads.

Decision rule: Bet on heads if $q > 1/2$ (i.e., $a' > b'$, equivalently $a + H > b + T$). Bet on tails if $q < 1/2$. If $q = 1/2$, do not bet.

For an even-money bet paying 2-for-1 (net odds of 1:1), the Kelly fraction is: $$f^{*} = 2q - 1 \quad \text{(if betting heads, i.e., } q > 1/2\text{)}$$ $$f^{*} = 1 - 2q \quad \text{(if betting tails, i.e., } q < 1/2\text{)}$$

More compactly: $f^{*} = |2q - 1|$, and bet on the side with posterior probability $> 1/2$.

Derivation: You wager fraction $f$ of your bankroll. If you win (probability $q$ for heads), your bankroll multiplies by $(1 + f)$. If you lose (probability $1 - q$), it multiplies by $(1 - f)$. The expected log growth rate is: $$G(f) = q \ln(1 + f) + (1 - q) \ln(1 - f)$$

Maximizing: $G'(f) = q/(1+f) - (1-q)/(1-f) = 0$, which gives $f^{*} = 2q - 1$ (assuming $q > 1/2$, so we are betting heads).

Expected log growth rate at the Kelly fraction: $$G(f^{*}) = q \ln(2q) + (1 - q) \ln(2(1-q))$$ $$= \ln 2 + q \ln q + (1-q) \ln(1-q)$$ $$= \ln 2 - \mathcal{H}(q)$$

where $\mathcal{H}(q) = -q \ln q - (1-q) \ln(1-q)$ is the binary entropy. The growth rate equals $\ln 2$ minus the entropy of your belief -- the more certain you are, the faster you grow.

Answer:

1. Posterior: $p \mid \text{data} \sim \text{Beta}(a+H, b+T)$. Posterior predictive for next flip being heads: $(a+H)/(a+b+H+T)$.

1. Posterior predictive for $h$ heads in $k$ flips: $\text{Beta-Binomial}(k, a+H, b+T)$.

1. Bet on whichever side has posterior probability $> 1/2$. Kelly fraction: $f^{*} = |2q - 1|$ where $q = (a+H)/(a+b+H+T)$. Expected log growth: $\ln 2 - \mathcal{H}(q)$.