Law of total probability

In probability theory, the law (or formula) of total probability is a fundamental rule relating marginal probabilities to conditional probabilities.

Statement

The law of total probability is the proposition that if $\left\{{B_n : n = 1, 2, 3, \ldots}\right\}$ is a finite or countably infinite partition of a sample space (in other words, a set of pairwise disjoint events whose union is the entire sample space) and each event $B_n$ is measurable, then for any event $A$ of the same probability space:

$\Pr(A)=\sum_n \Pr(A\cap B_n)\,$

or, alternatively,

$\Pr(A)=\sum_n \Pr(A\mid B_n)\Pr(B_n)\,$,

where, for any $n\,$ for which $\Pr(B_n) = 0 \,$ these terms are simply omitted from the summation, because $\Pr(A\mid B_n)\,$ is finite.

The summation can be interpreted as a weighted average, and consequently the marginal probability, $\Pr(A)$, is sometimes called "average probability"; "overall probability" is sometimes used in less formal writings.

The law of total probability can also be stated for conditional probabilities. Taking the $B_n$ as above, and assuming $X$ is not mutually exclusive with $A$ or any of the $B_n$:

$\Pr(A \mid X) = \sum_n \Pr(A \mid X \cap B_n) \Pr(B_n \mid X) \,$

Applications

One common application of the law is where the events coincide with a discrete random variable X taking each value in its range, i.e. $B_n$ is the event $X=x_n$. It follows that the probability of an event A is equal to the expected value of the conditional probabilities of A given $X=x_n$.^{[citation needed]} That is,

$\Pr(A)=\sum_n \Pr(A\mid X=x_n)\Pr(X=x_n) = \operatorname{E}_X[\Pr(A\mid X)] ,$

where Pr(A|X) is the conditional probability of A given X, and where E_X denotes the expectation with respect to the random variable X.^{[citation needed]}

This result can be generalized to continuous random variables (via continuous conditional density), and the expression becomes

$\Pr(A)= \operatorname{E}[\Pr(A\mid \mathcal{F}_X)],$

where $\mathcal{F}_X$ denotes the sigma-algebra generated by the random variable X.^{[citation needed]}

Other names

The term law of total probability is sometimes taken to mean the law of alternatives, which is a special case of the law of total probability applying to discrete random variables.^{[citation needed]} One author even uses the terminology "continuous law of alternatives" in the continuous case. This result is given by Grimmett and Welsh as the partition theorem, a name that they also give to the related law of total expectation.

See also

• Law of total expectation
• Law of total variance
• Law of total cumulance

References

• Introduction to Probability and Statistics by William Mendenhall, Robert J. Beaver, Barbara M. Beaver, Thomson Brooks/Cole, 2005, page 159.
• Theory of Statistics, by Mark J. Schervish, Springer, 1995.
• Schaum's Outline of Theory and Problems of Beginning Finite Mathematics, by John J. Schiller, Seymour Lipschutz, and R. Alu Srinivasan, McGraw-Hill Professional, 2005, page 116.
• A First Course in Stochastic Models, by H. C. Tijms, John Wiley and Sons, 2003, pages 431–432.
• An Intermediate Course in Probability, by Alan Gut, Springer, 1995, pages 5–6.