Almost surely

In probability theory, one says that an event happens almost surely (sometimes abbreviated as a.s.) if it happens with probability one. The concept is analogous to the concept of "almost everywhere" in measure theory. While there is no difference between almost surely and surely (that is, entirely certain to happen) in many basic probability experiments, the distinction is important in more complex cases relating to some sort of infinity. For instance, the term is often encountered in questions that involve infinite time, regularity properties or infinite-dimensional spaces such as function spaces. Basic examples of use include the law of large numbers (strong form) or continuity of Brownian paths.

Almost never describes the opposite of almost surely; an event which happens with probability zero happens almost never.

Formal definition

Let $(\Omega ,{\mathcal {F}},P)$ be a probability space. An event $E\in {\mathcal {F}}$ happens almost surely if $P[E]=1$. Equivalently, $E$ happens almost surely if the probability of $E$ not occurring is zero: $P[E^{C}]=0$. More generally, any event $E$ (not necessarily in ${\mathcal {F}}$) happens almost surely if $E^{C}$ is contained in a null set: a subset of some $N\in {\mathcal {F}}$ such that $P[N]=0$. The notion of almost sureness depends on the probability measure $P$. If it is necessary to emphasize this dependence, it is customary to say that the event $E$ occurs $P$-almost surely or almost surely $[P]$.

"Almost sure" versus "sure"

The difference between an event being almost sure and sure is the same as the subtle difference between something happening with probability 1 and happening always.

If an event is sure, then it will always happen, and no outcome not in this event can possibly occur. If an event is almost sure, then outcomes not in this event are theoretically possible; however, the probability of such an outcome occurring is smaller than any fixed positive probability, and therefore must be 0. Thus, one cannot definitively say that these outcomes will never occur, but can for most purposes assume this to be true.

Throwing dice

For example, image throwing a single die an infinite number of times. Although it is possible for a six to be rolled every single time, it is almost certain that this doesn't occur, and other numbers are rolled.

Tossing a coin

Consider the case where a coin is tossed. A coin has two sides, heads and tails, and therefore the event that "heads or tails is flipped" is a sure event. There can be no other result from such a coin.

Now consider the single "coin toss" probability space $(\{H,T\},2^{\{H,T\}},\mathbb {P} )$, where the event $\{\omega =H\}$ occurs if heads is flipped, and $\{\omega =T\}$ if tails. For this particular coin, assume the probability of flipping heads is $\mathbb {P} [\omega =H]=p\in (0,1)$ from which it follows that the complement event, flipping tails, has $\mathbb {P} [\omega =T]=1-p$.

Suppose we were to conduct an experiment where the coin is tossed repeatedly, and it is assumed each flip's outcome is independent of all the others. That is, they are i.i.d.. Define the sequence of random variables on the coin toss space, $\{X_{i}(\omega )\}_{i\in \mathbb {N} }$ where $X_{i}(\omega )=\omega _{i}$. i.e. each $X_{i}$ records the outcome of the $i$'th flip.

The event that every flip results in heads, yielding the sequence $\{H,H,H,\dots \}$, ad infinitum, is possible in some sense (it does not violate any physical or mathematical laws to suppose that tails never appears), but it is very, very improbable. In fact, the probability of tails never being flipped in an infinite series is zero. To see why, note that the i.i.d. assumption implies that the probability of flipping all heads over $n$ flips is simply $\mathbb {P} [X_{i}=H,\ i=1,2,\dots ,n]=\left(\mathbb {P} [X_{1}=H]\right)^{n}=p^{n}$. Letting $n\rightarrow \infty$ yields zero, since $p\in (0,1)$ by assumption. Note that the result is the same no matter how much we bias the coin towards heads, so long as we constrain $p$ to be greater than 0, and less than 1.

Thus, though we cannot definitely say tails will be flipped at least once, we can say there will almost surely be at least one tails in an infinite sequence of flips. (Note that given the statements made in this paragraph, any predefined infinitely long ordering, such as the digits of pi in base two with heads representing 1 and tails representing 0, would have zero-probability in an infinite series. This makes sense because there are an infinite number of total possibilities and $\scriptstyle \lim \limits _{n\to \infty }{\frac {1}{n}}=0$.)

However, if instead of an infinite number of flips we stop flipping after some finite time, say a million flips, then the all-heads sequence has non-zero probability. The all-heads sequence has probability $p^{1,000,000}\neq 0$, while the probability of getting at least one tails is $1-p^{1,000,000}$ and the event is no longer almost sure.

Asymptotically almost surely

In asymptotic analysis, one says that a property holds asymptotically almost surely (a.a.s.) if, over a sequence of sets, the probability converges to 1. For instance, a large number is asymptotically almost surely composite, by the prime number theorem; and in random graph theory, the statement "G(n,p_n) is connected" (where G(n,p) denotes the graphs on n vertices with edge probability p) is true a.a.s when p_n > ${\tfrac {(1+\epsilon )\ln n}{n}}$ for any ε > 0.

In number theory this is referred to as "almost all", as in "almost all numbers are composite". Similarly, in graph theory, this is sometimes referred to as "almost surely".

See also

• Convergence of random variables, for "almost sure convergence"
• Degenerate distribution, for "almost surely constant"
• Almost everywhere, the corresponding concept in measure theory
• Infinite monkey theorem, a theorem using the aforementioned terms.

References

• Rogers, L. C. G. (2000). Diffusions, Markov Processes, and Martingales. Vol. 1. Cambridge University Press. {{cite book}}: Unknown parameter |coauthors= ignored (|author= suggested) (help)
• Williams, David (1991). Probability with Martingales. Cambridge University Press.