Bayesian Posterior Calculation

Probability · Easy · Free problem

You have an unknown parameter $\theta$ with a prior distribution $P(\theta)$. You observe data $D$ and know the likelihood $P(D \mid \theta)$.

  1. Write down the posterior distribution $P(\theta \mid D)$ using Bayes' theorem.
  1. Suppose $\theta$ is the probability of heads for a coin, with a uniform prior $\theta \sim \text{Uniform}(0,1)$, and you observe $n = 10$ flips with $k = 7$ heads. Compute the posterior distribution and the posterior mean of $\theta$.
  1. What is the posterior predictive probability that the next flip is heads?

Hints

  1. Bayes' theorem says posterior is proportional to likelihood times prior -- think about what family the posterior belongs to.
  2. The uniform distribution on $[0,1]$ is $\text{Beta}(1,1)$, and the Beta family is conjugate to the Binomial likelihood.
  3. The posterior is $\text{Beta}(1 + k, 1 + n - k)$ and the posterior predictive for the next observation equals the posterior mean $\frac{1+k}{2+n}$.

Worked Solution

How to Think About It: Bayesian inference is how you update your beliefs when you see data. You start with a prior (what you believed before seeing anything), multiply by the likelihood (how probable the data is under each possible parameter value), and normalize. The result is the posterior -- your updated belief. The key practical move is to recognize conjugate pairs: when the prior and likelihood combine to give a posterior in the same family, the math stays clean.

Quick Estimate: With 7 heads out of 10 and a uniform prior, the posterior mean should be pulled toward $7/10 = 0.7$ but smoothed slightly toward $0.5$ by the prior. Since the uniform prior is $\text{Beta}(1,1)$, the posterior mean is $(1+7)/(1+1+10) = 8/12 \approx 0.667$. The posterior predictive for the next flip being heads is the same as the posterior mean (by exchangeability), so roughly $0.667$.

Approach: Use Bayes' theorem with the Beta-Binomial conjugate pair.

Formal Solution:

Part 1: Bayes' theorem states:

$P(\theta \mid D) = \frac{P(D \mid \theta) \, P(\theta)}{P(D)}$

where the marginal likelihood (evidence) is:

$P(D) = \int P(D \mid \theta') \, P(\theta') \, d\theta'$

Part 2: The uniform prior on $[0,1]$ is $\text{Beta}(1,1)$. The likelihood for $k = 7$ heads in $n = 10$ flips is:

$P(D \mid \theta) = \binom{10}{7} \theta^7 (1 - \theta)^3$

Since the Beta distribution is conjugate to the Binomial likelihood, the posterior is:

$\theta \mid D \sim \text{Beta}(1 + 7, \, 1 + 3) = \text{Beta}(8, 4)$

The posterior mean is:

$E[\theta \mid D] = \frac{8}{8 + 4} = \frac{2}{3}$

Part 3: By the Beta-Binomial conjugacy, the posterior predictive probability that the next flip is heads equals the posterior mean:

$P(X_{n+1} = H \mid D) = E[\theta \mid D] = \frac{2}{3}$

This follows because $P(X_{n+1} = H \mid D) = \int_0^1 \theta \cdot P(\theta \mid D) \, d\theta = E[\theta \mid D]$.

Answer: The posterior is $\text{Beta}(1+k, \, 1+n-k)$. For $n=10, k=7$: posterior $\text{Beta}(8,4)$, posterior mean

/3$, and the predictive probability of the next flip being heads is also /3$.

Intuition

Bayesian updating with a conjugate prior is the bread and butter of probabilistic reasoning in quant finance. The Beta-Binomial setup is the simplest example: you start with a belief about a probability, see binary outcomes, and the posterior just increments the Beta parameters by the number of successes and failures. The posterior mean is a weighted average of the prior mean and the observed frequency, with the weights determined by how much data you have relative to the strength of your prior.

In practice, this shows up whenever you are making a market on a binary event with limited data -- the posterior mean is your fair price. The key subtlety people miss: the posterior predictive accounts for parameter uncertainty, which fattens the tails compared to plugging in a point estimate. With very little data, the prior matters a lot; with abundant data, it washes out. Understanding this tension between prior and data is essential for any Bayesian problem in an interview.

Open the full interactive solver →