Total Probability Theorem

Suppose you want the probability that a widget is defective. You could try to count defective widgets directly — but what if the factory has multiple machines, each with a different defect rate, and you don’t know which machine made a given widget? You can’t just use one defect rate; you need to account for all of them.

Let’s label things: call the event we care about “$B$” (the widget is defective), and call the possible scenarios “$A_1, A_2, \ldots, A_n$” (machine 1, machine 2, …). We use different letters because $B$ and the $A_i$’s play different roles — $B$ is the event whose probability we want, and the $A_i$’s are the mutually exclusive cases that cover all possibilities. The total probability theorem says you can compute $P(B)$ by adding up each scenario’s contribution: the probability of that scenario times the probability of $B$ given that scenario. The result is a weighted average — scenarios that are more likely get more weight.

Partition

The theorem requires that the scenarios $A_1, A_2, \ldots, A_n$ form a partition of the sample space — mutually exclusive, collectively exhaustive events where every outcome belongs to exactly one $A_i$.

The Theorem

If $A_1, A_2, \ldots, A_n$ form a partition, then for any event $B$:

$$P(B) = \sum_{i=1}^{n} P(A_i) \cdot P(B | A_i)$$

Reading the formula: $P(A_i)$ is how likely scenario $i$ is. $P(B | A_i)$ is the probability of $B$ if scenario $i$ occurred. Their product $P(A_i) \cdot P(B | A_i)$ is the probability that scenario $i$ occurs and $B$ also occurs. Summing over all scenarios gives the total probability of $B$.

Derivation & Interpretation

Step 1: Break $B$ into pieces using the partition

Every outcome in $B$ must belong to at least one $A_i$ (the partition covers all of $\Omega$). And no outcome can belong to two different $A_i$’s (they’re mutually exclusive). So $B$ splits cleanly into non-overlapping pieces:

$$B = (A_1 \cap B) \;\cup\; (A_2 \cap B) \;\cup\; \cdots \cup\; (A_n \cap B)$$

The big union notation $\bigcup_{i=1}^n$ is just shorthand for this chain of “OR” — it means “the union of all these sets, from $i=1$ to $i=n$.”

$$B = \bigcup_{i=1}^n (A_i \cap B)$$

Think of it like slicing a pie: $B$ is the whole pie, and each $A_i \cap B$ is one slice. The slices don’t overlap and together make up the whole.

Step 2: From OR to ADD — why the union becomes a sum

Step 1 is a set equation — sets combine with union (OR). You can’t add sets. But once we take the probability of both sides, we’re dealing with numbers, and numbers add:

$$P(B) = P\!\left(\bigcup_{i=1}^n (A_i \cap B)\right) = \sum_{i=1}^n P(A_i \cap B)$$

Why can we turn OR into ADD? Because the pieces are disjoint — an outcome can’t be in both $A_1 \cap B$ and $A_2 \cap B$, since it can’t be in both $A_1$ and $A_2$. For disjoint events, the probability of their union equals the sum of their probabilities (axiom 3). Without disjointness, this step fails — you’d double-count outcomes that appear in multiple pieces.

This OR → ADD step is the whole reason the theorem works. The partition guarantees no overlap, so no double-counting.

Step 3: Apply the multiplication rule to each piece

By the multiplication rule, each piece $P(A_i \cap B) = P(A_i) \cdot P(B \mid A_i)$. Substituting:

$$P(B) = \sum_{i=1}^n P(A_i) \cdot P(B \mid A_i)$$

That’s the theorem. Three steps: slice → add → expand.

What the formula means

Each term $P(A_i) \cdot P(B \mid A_i)$ is one scenario’s contribution:

The total probability $P(B)$ is a weighted average — each scenario contributes proportionally to how often it occurs. A scenario that happens 60% of the time carries 60% of the weight, regardless of its conditional probability $P(B \mid A_i)$.

Example: Manufacturing

A factory has 3 machines producing widgets:

• Machine A: 50% of widgets, 2% defective
• Machine B: 30% of widgets, 4% defective
• Machine C: 20% of widgets, 5% defective

What’s the probability a widget is defective?

$$P(D) = 0.5 \times 0.02 + 0.3 \times 0.04 + 0.2 \times 0.05 = 0.033$$

The partition is $\{A, B, C\}$ — every widget comes from exactly one machine. Machine A contributes the most weight (0.5) because it produces the most widgets, even though its defect rate is the lowest.

Visual: Probability Tree

Sum defective paths: $0.01 + 0.012 + 0.01 = 0.033$

Special Case: Equally Likely Partition

When all $P(A_i) = \frac{1}{n}$, the formula simplifies to an average:

$$P(B) = \frac{1}{n} \sum_{i=1}^n P(B | A_i)$$