Stationary Distribution of a Candy Conveyor Belt Markov Chain
A conveyor belt dispenses two types of candy: chocolate (C) and jelly bean (J). The sequence is generated by a two-state Markov chain with the following transition probabilities:
$$P(J | C) = \frac{4}{5}, \quad P(C | C) = \frac{1}{5}$$ $$P(C | J) = \frac{2}{3}, \quad P(J | J) = \frac{1}{3}$$
Assuming the chain has been running long enough to reach stationarity, what fraction of candies are jelly beans?
Hints
- In a two-state Markov chain, the stationary distribution satisfies a simple flow balance: the long-run rate of transitions from C to J must equal the rate from J to C.
- Flow balance gives $\pi_C \cdot P(J|C) = \pi_J \cdot P(C|J)$, i.e., $\pi_C \times 4/5 = \pi_J \times 2/3$. Combined with $\pi_C + \pi_J = 1$, this is two equations in two unknowns.
- From $4\pi_C/5 = 2\pi_J/3$, get $\pi_C/\pi_J = 5/6$. With $\pi_C + \pi_J = 1$: $\pi_J = 6/11$.
Worked Solution
How to Think About It: In steady state, the rate of flow from C to J must equal the rate of flow from J to C -- otherwise the proportion of each type would drift over time. This balance condition is called the "flow balance" or "detailed balance" equation and it gives a very fast route to the answer. Before computing: chocolates transition to jelly beans with probability 4/5 (high rate leaving C), while jelly beans transition to chocolates with probability 2/3 (moderate rate leaving J). Since chocolates leave C quickly, C should be rare in steady state. Rough guess: maybe 30% chocolate, 70% jelly bean.
Quick Estimate: Let $\pi_J$ be the steady-state fraction of jelly beans. The flow out of the C state equals $\pi_C \times P(J|C) = \pi_C \times 4/5$, and the flow into the C state from J equals $\pi_J \times P(C|J) = \pi_J \times 2/3$. Setting these equal and using $\pi_C + \pi_J = 1$ should give us $\pi_J$ directly.
Approach: Solve the stationary distribution equations $\pi = \pi P$ (or equivalently, flow balance).
Formal Solution:
The transition matrix is: $$P = \begin{pmatrix} 1/5 & 4/5 \\ 2/3 & 1/3 \end{pmatrix}$$
where rows are current state (C, J) and columns are next state (C, J).
The stationary distribution $\pi = (\pi_C, \pi_J)$ satisfies $\pi P = \pi$ with $\pi_C + \pi_J = 1$.
Flow balance (equivalent and faster for two states): the rate of C-to-J transitions equals the rate of J-to-C transitions: $$\pi_C \cdot P(J|C) = \pi_J \cdot P(C|J)$$ $$\pi_C \cdot \frac{4}{5} = \pi_J \cdot \frac{2}{3}$$
Substituting $\pi_C = 1 - \pi_J$: $$(1 - \pi_J) \cdot \frac{4}{5} = \pi_J \cdot \frac{2}{3}$$ $$\frac{4}{5} - \frac{4}{5}\pi_J = \frac{2}{3}\pi_J$$ $$\frac{4}{5} = \pi_J \left(\frac{2}{3} + \frac{4}{5}\right) = \pi_J \cdot \frac{10 + 12}{15} = \pi_J \cdot \frac{22}{15}$$ $$\pi_J = \frac{4}{5} \cdot \frac{15}{22} = \frac{60}{110} = \frac{6}{11}$$
And $\pi_C = 1 - 6/11 = 5/11$.
Verification: Check $\pi P = \pi$: $$\pi_C^{\text{new}} = \pi_C \cdot \frac{1}{5} + \pi_J \cdot \frac{2}{3} = \frac{5}{11} \cdot \frac{1}{5} + \frac{6}{11} \cdot \frac{2}{3} = \frac{1}{11} + \frac{4}{11} = \frac{5}{11} \checkmark$$
$$\boxed{\pi_J = \frac{6}{11} \approx 0.545}$$
About 54.5% of candies are jelly beans in the long run.
Answer: The stationary fraction of jelly beans is $6/11 \approx 54.5\%$.
Intuition
For a two-state Markov chain, the stationary distribution is determined entirely by the two off-diagonal transition rates. The formula is immediate from flow balance: $\pi_J / \pi_C = P(J|C) / P(C|J)$. In words, the steady-state ratio of J to C is equal to the ratio of the rates at which you leave C for J versus leave J for C. Here, $4/5 \div 2/3 = 6/5$, so there are 6 parts J for every 5 parts C, giving $\pi_J = 6/11$.
This flow balance principle extends to reversible Markov chains of any size via the detailed balance equations $\pi_i P_{ij} = \pi_j P_{ji}$. For non-reversible chains you need the full system $\pi P = \pi$, but for two states detailed balance always holds (any two-state chain is reversible). In finance, Markov chains with stationary distributions model regime-switching processes -- for instance, a market that alternates between a bull state and a bear state. The fraction of time in each regime is determined by exactly this calculation, and it governs long-run average returns, volatility, and risk metrics.