Chain rule (probability)

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search

In probability theory, the chain rule (also called the general product rule[1][2]) permits the calculation of any member of the joint distribution of a set of random variables using only conditional probabilities. The rule is useful in the study of Bayesian networks, which describe a probability distribution in terms of conditional probabilities.

Chain rule for events[edit]

Two events[edit]

The chain rule for two random events and says

.

Example[edit]

This rule is illustrated in the following example. Urn 1 has 1 black ball and 2 white balls and Urn 2 has 1 black ball and 3 white balls. Suppose we pick an urn at random and then select a ball from that urn. Let event be choosing the first urn: . Let event be the chance we choose a white ball. The chance of choosing a white ball, given that we have chosen the first urn, is . Event would be their intersection: choosing the first urn and a white ball from it. The probability can be found by the chain rule for probability:

.

More than two events[edit]

For more than two events the chain rule extends to the formula

which by induction may be turned into

.

Example[edit]

With four events (), the chain rule is

Chain rule for random variables[edit]

Two random variables[edit]

For two random variables , to find the joint distribution, we can apply the definition of conditional probability to obtain:

More than two random variables[edit]

Consider an indexed collection of random variables . To find the value of this member of the joint distribution, we can apply the definition of conditional probability to obtain:

Repeating this process with each final term creates the product:

Example[edit]

With four variables (), the chain rule produces this product of conditional probabilities:

Footnotes[edit]

References[edit]

  • Schum, David A. (1994). The Evidential Foundations of Probabilistic Reasoning. Northwestern University Press. p. 49. ISBN 978-0-8101-1821-8.CS1 maint: ref=harv (link)
  • Klugh, Henry E. (2013). Statistics: The Essentials for Research (3rd ed.). Psychology Press. p. 149. ISBN 1-134-92862-9.CS1 maint: ref=harv (link)
  • Russell, Stuart J.; Norvig, Peter (2003), Artificial Intelligence: A Modern Approach (2nd ed.), Upper Saddle River, New Jersey: Prentice Hall, ISBN 0-13-790395-2, p. 496.
  • "The Chain Rule of Probability", developerWorks, Nov 3, 2012.