Bayesian Modeling of Rare Events

Worked Solution

How to Think About It: This is a design/methodology question -- the interviewer wants to see that you can set up a proper Bayesian model from scratch and understand the practical implications. The core challenge with rare events is that you have very few observations, so the prior matters a lot. A frequentist approach (just use $k/n$) is fragile when $k$ is small -- you might observe 0 events and conclude the probability is exactly zero. Bayesian shrinkage toward a sensible prior fixes this.

Key Insight: Use a conjugate prior so the posterior is analytically tractable. For a binary event with Binomial likelihood, the conjugate prior is Beta. The posterior mean automatically shrinks the MLE toward the prior mean, which is exactly the regularization you want when data is sparse.

The Method:

Step 1 -- Prior: Place a $\text{Beta}(\alpha, \beta)$ prior on the event probability $p$. For a rare event, choose $\alpha$ small and $\beta$ large so the prior mean $\alpha / (\alpha + \beta)$ reflects your belief about the event rate. For example, $\text{Beta}(1, 99)$ centers the prior at $p = 0.01$ (1% event rate) with moderate uncertainty. If you have stronger prior information, use $\text{Beta}(2, 198)$ for the same mean but tighter concentration.

Step 2 -- Likelihood: Given $n$ independent trials with $k$ events observed:

$L(p \mid k, n) = \binom{n}{k} p^k (1 - p)^{n-k} \propto p^k (1 - p)^{n-k}$

Step 3 -- Posterior: By conjugacy, the posterior is:

$p \mid k \sim \text{Beta}(\alpha + k, \, \beta + n - k)$

The posterior mean is:

$\hat{p}_{\text{Bayes}} = \frac{\alpha + k}{\alpha + \beta + n}$

This is a weighted average of the prior mean $\alpha/(\alpha + \beta)$ and the MLE $k/n$. When $n$ is large relative to $\alpha + \beta$, the data dominates and $\hat{p}_{\text{Bayes}} \approx k/n$. When data is sparse (small $n$, small $k$), the prior provides regularization.

Step 4 -- Poisson Alternative: For very rare events where $p$ is tiny and $n$ is large, a Poisson model is more natural. Model the count of events as $k \sim \text{Poisson}(n\lambda)$ where $\lambda$ is the event rate per trial. Use a conjugate $\text{Gamma}(\alpha, \beta)$ prior on $\lambda$. The posterior is:

$\lambda \mid k \sim \text{Gamma}(\alpha + k, \, \beta + n)$

The Poisson-Gamma model is preferred when you are counting rare events over a large exposure (e.g., defaults per 10,000 loans, crashes per trading day).

Practical Considerations: - Prior sensitivity: with few events, the posterior is sensitive to the prior. Run a sensitivity analysis by varying $\alpha$ and $\beta$. - The 95% credible interval $[p_{0.025}, \, p_{0.975}]$ from the Beta posterior is often more useful than the point estimate for risk management. - If you observe $k = 0$ events, the MLE is $\hat{p} = 0$, which is clearly wrong for risk purposes. The Bayesian estimate $\alpha / (\alpha + \beta + n)$ remains positive -- this is a major practical advantage.

Answer: Use a Beta-Binomial conjugate model with a prior that encodes rarity (small $\alpha$, large $\beta$). The posterior is $\text{Beta}(\alpha + k, \beta + n - k)$, and the posterior mean shrinks the MLE toward the prior mean. For very rare events with large exposure, switch to a Poisson-Gamma model.