Win Probability in a Streak-Dependent Slot Machine

Probability · Medium · Free problem

You play a slot machine with a twist: your win probability depends on what happened last turn.

  • On your first turn, you win with probability $\frac{1}{2}$.
  • After the first turn, if you lost the previous turn, you win with probability $\frac{1}{2}$.
  • If you won the previous turn, you win with a lower probability $0 < p < \frac{1}{2}$.

Let $p_n$ be the probability that you win on the $n$th turn.

  1. Find a recurrence relation for $p_n$ in terms of $p_{n-1}$.
  2. Solve the recurrence to get a closed-form expression for $p_n$.
  3. Evaluate $p_4$ when $p = \frac{1}{3}$.

Hints

  1. Condition on whether you won or lost on the previous turn and apply the law of total probability to write $p_n$ in terms of $p_{n-1}$.
  2. You should get a first-order linear recurrence $p_n = (p - 1/2)\,p_{n-1} + 1/2$. Try differencing consecutive terms to eliminate the constant and reduce to a geometric sequence.
  3. After differencing, you get $p_n - p_{n-1} = (p - 1/2)^{n-2} \cdot (2p-1)/4$. Telescope this sum and apply the geometric series formula to get a closed form.

Worked Solution

How to Think About It: This is a Markov chain problem with two states -- "just won" and "just lost" -- and your transition probabilities depend on which state you are in. The key modeling step is to realize that $p_n$ (the unconditional win probability on turn $n$) mixes over these two states using the law of total probability. Once you write that down, you get a first-order linear recurrence, which is bread and butter.

Before diving into the algebra, get a feel for what happens. On turn 1 you win with probability

/2$. If you win (prob
/2$), your next win rate drops to $p < 1/2$ -- the machine punishes streaks. If you lose (prob
/2$), you stay at
/2$. So the machine has a built-in mean-reversion: winning makes you less likely to win next time, pulling $p_n$ below
/2$ over time.

Quick Estimate: With $p = 1/3$, let us just iterate. $p_1 = 1/2$. For $p_2$: you won last turn with prob

/2$ (win rate
/3$) or lost with prob
/2$ (win rate
/2$), so $p_2 = (1/2)(1/3) + (1/2)(1/2) = 1/6 + 1/4 = 5/12 \approx 0.417$. Already below
/2$ -- the streak penalty is pulling you down. Continuing: $p_3 = (1/3)(5/12) + (1/2)(7/12) = 5/36 + 7/24 = 31/72 \approx 0.431$. And $p_4 = (1/3)(31/72) + (1/2)(41/72) = 31/216 + 41/144 = 185/432 \approx 0.428$. The values are oscillating and converging toward some steady state. In the long run, solving $p_{\infty} = p \cdot p_{\infty} + (1/2)(1 - p_{\infty})$ gives $p_{\infty} = 1/(3 - 2p)$, which for $p = 1/3$ is $3/7 \approx 0.429$. Our $p_4 \approx 0.428$ is already close.

Approach: Condition on the previous outcome to get a linear recurrence, then solve it by differencing to eliminate the constant term.

Formal Solution:

By the law of total probability, conditioning on whether you won or lost on turn $n-1$:

$p_n = p \cdot p_{n-1} + \frac{1}{2}(1 - p_{n-1}) = \left(p - \frac{1}{2}\right) p_{n-1} + \frac{1}{2}$

This is a first-order linear recurrence with constant coefficients. To solve it, define $r_n = p_n - p_{n-1}$ and difference consecutive equations:

$r_n = p_n - p_{n-1} = \left(p - \frac{1}{2}\right)(p_{n-1} - p_{n-2}) = \left(p - \frac{1}{2}\right) r_{n-1}$

So $r_n$ is a geometric sequence with ratio $p - 1/2$. The base case: $p_1 = 1/2$ and $p_2 = p \cdot (1/2) + (1/2)(1/2) = (2p + 1)/4$, giving $r_2 = p_2 - p_1 = (2p - 1)/4$.

Therefore:

$r_n = \frac{2p - 1}{4} \left(p - \frac{1}{2}\right)^{n-2}$

Telescoping:

$p_n = p_1 + \sum_{k=2}^{n} r_k = \frac{1}{2} + \frac{2p-1}{4} \sum_{j=0}^{n-2} \left(p - \frac{1}{2}\right)^j$

Applying the geometric series formula with common ratio $p - 1/2$ (which satisfies $|p - 1/2| < 1/2 < 1$):

$p_n = \frac{1}{2} + \frac{2p - 1}{4} \cdot \frac{1 - (p - 1/2)^{n-1}}{1 - (p - 1/2)}$

Simplifying the denominator

- (p - 1/2) = 3/2 - p = (3 - 2p)/2$:

$p_n = \frac{1}{2} + \frac{(2p-1)}{4} \cdot \frac{2[1 - (p - 1/2)^{n-1}]}{3 - 2p}$

Note that

p - 1 = 2(p - 1/2)$, so:

$p_n = \frac{1}{2} + \frac{(p - 1/2)[1 - (p-1/2)^{n-1}]}{3 - 2p} = \frac{1}{2} + \frac{(p - 1/2) - (p - 1/2)^n}{3 - 2p}$

Combining over a common denominator:

$p_n = \frac{(3-2p)/2 + (p - 1/2) - (p-1/2)^n}{3 - 2p} = \frac{1 - (p - 1/2)^n}{3 - 2p}$

For $p = 1/3$ and $n = 4$:

$p_4 = \frac{1 - (-1/6)^4}{7/3} = \frac{1 - 1/1296}{7/3} = \frac{1295/1296}{7/3} = \frac{1295 \times 3}{1296 \times 7} = \frac{3885}{9072} = \frac{185}{432}$

Answer: The closed-form solution is $p_n = \dfrac{1 - (p - 1/2)^n}{3 - 2p}$, and for $p = 1/3$, $n = 4$:

$p_4 = \frac{185}{432} \approx 0.428$

Intuition

This is a two-state Markov chain in disguise. The machine has a "hot hand" penalty: winning drops your next-turn probability from

/2$ down to $p$. This creates mean-reversion in $p_n$ -- after a win, the next win is harder, so $p_n$ oscillates and converges to a steady state $p_{\infty} = 1/(3 - 2p)$, which is always below
/2$ when $p < 1/2$. The closed form $p_n = [1 - (p - 1/2)^n]/(3 - 2p)$ makes the convergence rate explicit: the gap from steady state decays geometrically at rate $|p - 1/2|$.

The broader lesson is that whenever you see state-dependent transition probabilities, you should immediately think Markov chain and condition on the previous state. The differencing trick for solving first-order linear recurrences with constant coefficients is a workhorse technique -- it shows up in everything from random walks to pricing path-dependent options. In practice, the ability to quickly iterate a recurrence numerically (as in the quick estimate above) is just as valuable as finding the closed form, because it gives you an instant sanity check.

Open the full interactive solver →