Probability theory

Probability theory is the mathematical study of phenomena characterized by randomness or uncertainty.

More precisely, probability is used for modelling situations when the result of an experiment, realized under the same circumstances, produces different results (typically throwing a dice or a coin). Mathematicians and actuaries think of probabilities as numbers in the closed interval from 0 to 1 assigned to "events" whose occurrence or failure to occur is random. Probabilities $P(A)$ are assigned to events $A$ according to the probability axioms.

The probability that an event $A$ occurs given the known occurrence of an event $B$ is the conditional probability of $A$ given $B$; its numerical value is $P(A\cap B)/P(B)$ (as long as $P(B)$ is nonzero). If the conditional probability of $A$ given $B$ is the same as the ("unconditional") probability of $A$, then $A$ and $B$ are said to be independent events. That this relation between $A$ and $B$ is symmetric may be seen more readily by realizing that it is the same as saying $P(A\cap B)=P(A)P(B)$ when A and B are independent events.

Two crucial concepts in the theory of probability are those of a random variable and of the probability distribution of a random variable; see those articles for more information.

A somewhat more abstract view of probability

Mathematicians usually take probability theory to be the study of probability spaces and random variables — an approach introduced by Kolmogorov in the 1930s. A probability space is a triple $(\Omega ,{\mathcal {F}},P)$, where

• $\Omega$ is a non-empty set, sometimes called the "sample space," each of whose members is thought of as a potential outcome of a random experiment. For example, if 100 voters are to be drawn randomly from among all voters in California and asked whom they will vote for governor, then the set of all sequences of 100 Californian voters would be the sample space Ω.

• ${\mathcal {F}}$ is a σ-algebra of subsets of $\Omega$ - its members are called "events." For example the set of all sequences of 100 Californian voters in which at least 60 will vote for Schwarzenegger is identified with the "event" that at least 60 of the 100 chosen voters will so vote. To say that ${\mathcal {F}}$ is a σ-algebra implies per definition that it contains $\Omega$, that the complement of any event is an event, and that the union of any (finite or countably infinite) sequence of events is an event. So for this example ${\mathcal {F}}$ contains: (1) the set of all sequences of 100 where at least 60 vote for Schwarzenegger; (2) the set of all sequences of 100 where fewer than 60 vote for Schwarzenegger (the converse of (1)); (3) the sample space Ω as above; and (4) the empty set.

• $P$ is a probability measure on ${\mathcal {F}}$, i.e., a measure such that $P(\Omega )=1$.

It is important to note that $P$ is a function defined on ${\mathcal {F}}$ and not on $\Omega$, and often not on the complete powerset ${\mathcal {F}}=\mathbb {P} (\Omega )$ either. Not every set of outcomes is an event.

If $\Omega$ is denumerable we almost always define ${\mathcal {F}}$ as the power set of $\Omega$, i.e ${\mathcal {F}}=\mathbb {P} (\Omega )$ which is trivially a σ-algebra and the biggest one we can create using $\Omega$. In a discrete space we can therefore omit ${\mathcal {F}}$ and just write $(\Omega ,P)$ to define it. If on the other hand $\Omega$ is non-denumerable and we use ${\mathcal {F}}=\mathbb {P} (\Omega )$ we get into trouble defining our probability measure $P$ because ${\mathcal {F}}$ is too 'huge', i.e. there will often be sets to which it will be impossible to assign a unique measure, giving rise to problems like the Banach–Tarski paradox. So we have to use a smaller σ-algebra ${\mathcal {F}}$ (e.g. the Borel algebra of $\Omega$, which is the smallest σ-algebra that makes all open sets measurable).

A random variable $X$ is a measurable function on $\Omega$. For example, the number of voters who will vote for Schwarzenegger in the aforementioned sample of 100 is a random variable.

If $X$ is any random variable, the notation $P(X\geq 60)$, is shorthand for $P(\{\omega \in \Omega \mid X(\omega )\geq 60\})$, assuming that "$X\geq 60$" is an "event."

For an algebraic alternative to Kolmogorov's approach, see algebra of random variables.

Philosophy of application of probability

There are different ways to interpret probability. Frequentists will assign probabilities only to events that are random, i.e., random variables, that are outcomes of actual or theoretical experiments. On the other hand, Bayesians assign probabilities to propositions that are uncertain according either to subjective degrees of belief in their truth, or to logically justifiable degrees of belief in their truth. Among statisticians and philosophers, many more distinctions are drawn beyond this subjective/objective divide. See the article on interpretations of probability at the Stanford Encyclopedia of Philosophy: [1].

A Bayesian may assign a probability to the proposition that 'there was life on Mars a billion years ago', since that is uncertain, whereas a frequentist would not assign probabilities to statements at all. A frequentist is actually unable to technically interpret such uses of the probability concept, even though 'probability' is often used in this way in colloquial speech. Frequentists only assign probabilities to outcomes of well defined random experiments, that is, where there is a defined sample space as defined above in the theory section. For another illustration of the differences see the two envelopes problem.

Situations do arise where probability theory is somewhat lacking. One method of attempting to circumvent this indeterminancy is the theory of super-probability, in which situations are given integer values greater than 1. This is an extension of the multi-dimensional space intrinsic to M-theory and modern theoretical physics. <Popular Culture> The probability theory is used by "L" in the movie adaptation of the manga Death Note to try to catch the killer "Kira".

See also

• probabilistic logic
• credence (probability theory)
• glossary of probability and statistics
• list of probability topics
• list of statistical topics
• list of publications in statistics
• predictive modelling
• fuzzy measure theory
• probability axioms
• probability distribution
• expected value
• likelihood function
• random variable
• sample space
• variance
• statistical independence
• notation in probability
• possibility theory
• Pierre de Fermat
• Blaise Pascal

Bibliography

• Pierre Simon de Laplace (1812) Analytical Theory of Probability

The first major treatise blending calculus with probability theory, originally in French: Théorie Analytique des Probabilités.

• Andrei Nikolajevich Kolmogorov (1950) Foundations of the Theory of Probability

The modern measure-theoretic foundation of probability theory; the original German version (Grundbegriffe der Wahrscheinlichkeitrechnung) appeared in 1933.

• Harold Jeffreys (1939) The Theory of Probability

An empiricist, Bayesian approach to the foundations of probability theory.

• Edward Nelson (1987) Radically Elementary Probability Theory

Discrete foundations of probability theory, based on nonstandard analysis and internal set theory. downloadable. http://www.math.princeton.edu/~nelson/books.html

• Patrick Billingsley: Probability and Measure, John Wiley and Sons, New York, Toronto, London, 1979.

• Henk Tijms (2004) Understanding Probability

A lively introduction to probability theory for the beginner, Cambridge Univ. Press.