Roll Until You Stop or Hit a 6: Optimal Strategy and Fair Price
A casino offers the following game with a fair six-sided die. You may roll as many times as you like. Each roll of $1, 2, 3, 4$ or $5$ adds that many dollars to your running total. A roll of $6$ wipes out your entire running total and ends the game immediately. After any roll you may choose to stop and walk away with your running total.
You are risk-neutral.
(a) What is the optimal strategy: when should you stop rolling?
(b) What is the most you should be willing to pay to play this game?
Hints
- With a running total of $n$, compare stopping (worth $n$) with rolling exactly once more and then stopping: the one-step value is $\tfrac{1}{6}(n+1) + \cdots + \tfrac{1}{6}(n+5) + \tfrac{1}{6}\cdot 0 = \tfrac{5n}{6} + \tfrac{5}{2}$.
- $\tfrac{5n}{6} + \tfrac{5}{2} > n$ exactly when $n < 15$. Argue that once stopping beats one more roll it also beats any continuation, so the optimal rule is a threshold.
- Define $f(n)$ as the value of the game with running total $n$: $f(n) = n$ for $n \ge 15$ and $f(n) = \tfrac{1}{6}\sum_{i=1}^{5} f(n+i)$ for $n \le 14$. Compute backward down to $f(0)$.
Worked Solution
How to Think About It: This is a dynamic programming / optimal stopping problem. The state is the running total $n$. At each state you compare the value of stopping ($n$) with the expected value of continuing optimally. The trick is that the one-step comparison "stop vs. roll once more and stop" already identifies the threshold, and then the values below the threshold follow by backward recursion.
Quick Estimate: Rolling once more from total $n$ adds $3$ on average on the five safe faces but forfeits $n$ with probability $1/6$: the expected one-step change is $\tfrac{5}{6}\cdot 3 - \tfrac{1}{6}\,n = 2.5 - n/6$, which is positive only while $n < 15$. So the threshold is $15$, and starting from zero you will typically survive four or five rolls, so the game should be worth roughly $\$6$.
Formal Solution:
Part (a): Optimal strategy
*Step 1 -- One-step lookahead.* From total $n$, rolling once and then stopping is worth
$$\frac{1}{6}\sum_{i=1}^{5}(n + i) + \frac{1}{6}\cdot 0 = \frac{5n}{6} + \frac{5}{2}.$$
This exceeds $n$ iff $n < 15$. So at $n \ge 15$ even a single further roll is unfavorable.
*Step 2 -- Threshold is optimal.* Let $f(n)$ be the optimal value from total $n$. Suppose $n \ge 15$ and consider any continuation strategy. Its terminal total is $n + G$ on the no-6 event, where $G \ge 0$ is the gain, and $0$ if a 6 occurs first. Every additional roll from a total $m \ge 15$ has negative expected one-step gain, and the same holds inductively for any sequence of rolls, so no continuation from $n \ge 15$ beats stopping: $f(n) = n$ for $n \ge 15$. Formally, the process $M_k = \text{(current total)}$ under any rolling policy is a supermartingale once the total is at least 15, so by optional stopping its expected terminal value is at most $n$. For $n \le 14$ one more roll strictly improves on stopping, so the optimal policy is:
$$\text{keep rolling while the total is } \le 14; \text{ stop as soon as it is } \ge 15.$$
Part (b): Value of the game
*Step 3 -- Backward recursion.* With $f(n) = n$ for $n \ge 15$ and
$$f(n) = \frac{1}{6}\sum_{i=1}^{5} f(n+i), \qquad n \le 14,$$
compute downward:
| $n$ | $14$ | $13$ | $12$ | $11$ | $10$ | $9$ | $8$ | $7$ | |---|---|---|---|---|---|---|---|---| | $f(n)$ | $14.167$ | $13.361$ | $12.588$ | $11.853$ | $11.161$ | $10.522$ | $9.914$ | $9.340$ |
| $n$ | $6$ | $5$ | $4$ | $3$ | $2$ | $1$ | $0$ | |---|---|---|---|---|---|---|---| | $f(n)$ | $8.798$ | $8.289$ | $7.811$ | $7.359$ | $6.933$ | $6.532$ | $6.154$ |
For example $f(14) = \tfrac{1}{6}(15 + 16 + 17 + 18 + 19) = \tfrac{85}{6} \approx 14.167$ and $f(13) = \tfrac{1}{6}(f(14) + 15 + 16 + 17 + 18) = \tfrac{481}{36} \approx 13.361$.
*Step 4 -- Fair price.* The value of the game at the start is $f(0) \approx 6.15$ (exactly $2893395172951/470184984576$). A risk-neutral player should pay at most about $\$6.15$.
Answer: (a) Keep rolling while your total is at most $\$14$; stop as soon as it reaches $\$15$ or more. (b) The game is worth $f(0) \approx \$6.15$, so pay at most that.
Intuition
Every extra roll risks one sixth of everything you have banked in exchange for an average gain of $3$ dollars, so rolling is worth it only while $n/6 < 2.5$, i.e. while your total is at most $14$. That gives a threshold rule (stop at $15$ or more), and backward induction from the threshold values the empty-handed start at about $\$6.15$. This is a miniature optimal-stopping problem with a bang-bang solution: the same "compare the marginal expected gain with the marginal expected loss of what you already hold" logic prices early exercise of American options and decides when to close a profitable position.