Convergence of measures

In mathematics, more specifically measure theory, there are various notions of the convergence of measures. For an intuitive general sense of what is meant by convergence in measure, consider a sequence of measures $\mu_n$ on a space, sharing a common collection of measurable sets. Such a sequence might represent an attempt to construct 'better and better' approximations to a desired measure $\mu$ that is difficult to obtain directly. The meaning of 'better and better' is subject to all the usual caveats for taking limits; for any error tolerance $\epsilon > 0$ we require there be $N$ sufficiently large for $n \geq N$ to ensure the 'difference' between $\mu_n$ and $\mu$ is smaller than $\epsilon$. Various notions of convergence specify precisely what the word 'difference' should mean in that description; these notions are not equivalent to one another, and vary in strength.

Three of the most common notions of convergence are described below.

Informal descriptions

This section attempts to provide a rough intuitive description of three notions of convergence, using terminology developed in calculus courses; this section is necessarily imprecise as well as inexact, and the reader should refer to the formal clarifications in subsequent sections. In particular, the descriptions here do not address the possibility that $\mu_n(A)$ or $\mu(A)$ could be infinite or zero.

The various notions of weak convergence formalize the assertion that the 'average value' of each 'nice' function should converge:

${1 \over \mu_n(A)}\int_A f d\mu_n \to {1 \over \mu(A)}\int_A f d\mu$

To formalize this requires a careful specification of the set of functions under consideration, the domain of integration $A$, and of what is meant by average value (see expectation). This notion treats functions $f$ completely independently of one another, i.e. the $N$ above varies with $f$.

The notion of strong convergence formalizes the assertion that the measure of each measurable set should converge:

$\mu_n(A) \to \mu(A)$

Intuitively, considering integrals of 'nice' functions, this notion provides more uniformity than weak convergence: for any $\epsilon > 0$ and any upper bound $M > 0$, we can find an $N$ so that $n \geq N$ ensures

$\left|\int_A f d\mu_n - \int_A f d\mu\right| < M\epsilon$

for all integrable functions $f$ with $\left|f\right|$ bounded by $M$. (The definition of strong convergence does not require one to specify a set of measurable functions, so the interpretation given here might not apply.) This notion of convergence still allows $N$ to vary with the set $A$.

The notion of total variation convergence formalizes the assertion that the measure of all measurable sets should converge uniformly, i.e. we do not allow the $N$ described above to depend upon which set $A$ we are measuring. (This is only a very rough description of total variation convergence, as additional technical care is necessary; see below.)

Total variation convergence of measures

This is the strongest notion of convergence shown on this page and is defined as follows. Let $(X, \mathcal{F})$ be a measurable space. The total variation distance between two (positive) measures $\mu$ and $\nu$ is then given by

$\|\mu- \nu\|_{TV} = \sup \Bigl\{\int_X f(x) (\mu-\nu)(dx) \Big| f\colon X \to [-1,1] \Bigr\}.$

If $\mu$ and $\nu$ are both probability measures, then the total variation distance is also given by

$\|\mu- \nu\|_{TV} = 2\cdot\sup_{A\in \mathcal{F}} | \mu (A) - \nu (A) | .$

The equivalence between these two definitions can be seen as a particular case of the Monge-Kantorovich duality. From the two definitions above, it is clear that the total variation distance between probability measures is always between 0 and 2.

To illustrate the meaning of the total variation distance, consider the following thought experiment. Assume that we are given two probability measures $\mu$ and $\nu$, as well as a random variable $X$. We know that $X$ has law either $\mu$ or $\nu$, but we do not know which one of the two. Assume now that we are given one single sample distributed according to the law of $X$ and that we are then asked to guess which one of the two distributions describes that law. The quantity

${2+\|\mu-\nu\|_{TV} \over 4}$

then provides a sharp upper bound on the probability that our guess is correct.

Strong convergence of measures

For $(X, \mathcal{F})$ a measurable space, a sequence $\mu_n$ is said to converge strongly to a limit $\mu$ if

$\lim_{n \to \infty} \mu_n(A) = \mu(A)$

for every set in $\mathcal{F}$.

For example, as a consequence of the Riemann–Lebesgue lemma, the sequence $\mu_n$ of measures on the interval [-1,1] given by $\mu_n(dx) = (1+ \sin(nx))\,dx$ converges strongly to Lebesgue measure, but it does not converge in total variation.

Weak convergence of measures

In mathematics and statistics, weak convergence (also known as narrow convergence or weak-* convergence which is a more appropriate name from the point of view of functional analysis but less frequently used) is one of many types of convergence relating to the convergence of measures. It depends on a topology on the underlying space and thus is not a purely measure theoretic notion.

There are several equivalent definitions of weak convergence of a sequence of measures, some of which are (apparently) more general than others. The equivalence of these conditions is sometimes known as the portmanteau theorem.

Definition. Let S be a metric space with its Borel σ-algebra Σ. We say that a sequence of probability measures on (S, Σ), P_n, n = 1, 2, ..., converges weakly to the probability measure P, and write

$P_n\Rightarrow P$

if any of the following equivalent conditions is true:

• E_nƒ → Eƒ for all bounded, continuous functions ƒ;
• E_nƒ → Eƒ for all bounded and Lipschitz functions ƒ;
• limsup E_nƒ ≤ Eƒ for every upper semi-continuous function ƒ bounded from above;
• liminf E_nƒ ≥ Eƒ for every lower semi-continuous function ƒ bounded from below;
• limsup P_n(C) ≤ P(C) for all closed sets C of space S;
• liminf P_n(U) ≥ P(U) for all open sets U of space S;
• lim P_n(A) = P(A) for all continuity sets A of measure P.

In the case S = R with its usual topology, if F_n, F denote the cumulative distribution functions of the measures P_n, P respectively, then P_n converges weakly to P if and only if lim_n→∞ F_n(x) = F(x) for all points x ∈ R at which F is continuous.

For example, the sequence where P_n is the Dirac measure located at 1/n converges weakly to the Dirac measure located at 0 (if we view these as measures on R with the usual topology), but it does not converge strongly. This is intuitively clear: we only know that 1/n is "close" to 0 because of the topology of R.

This definition of weak convergence can be extended for S any metrizable topological space. It also defines a weak topology on P(S), the set of all probability measures defined on (S, Σ). The weak topology is generated by the following basis of open sets:

$\left\{ U_{\phi, x, \delta} \,\left|\, \begin{array}{c} \phi \colon S \to \mathbb{R} \text{ is bounded and continuous,} \\ x \in \mathbb{R} \text{ and } \delta > 0 \end{array} \right. \right\},$

where

$U_{\phi, x, \delta} := \left\{ \mu \in \boldsymbol{P}(S) \,\left|\, \left| \int_{S} \phi \, \mathrm{d} \mu - x \right| < \delta \right. \right\}.$

If S is also separable, then P(S) is metrizable and separable, for example by the Lévy–Prokhorov metric, if S is also compact or Polish, so is P(S).

If S is separable, it naturally embeds into P(S) as the (closed) set of dirac measures, and its convex hull is dense.

There are many "arrow notations" for this kind of convergence: the most frequently used are $P_{n} \Rightarrow P$, $P_{n} \rightharpoonup P$ and $P_{n} \xrightarrow{\mathcal{D}} P.$.

Weak convergence of random variables

Main article: Convergence of random variables

Let $(\Omega, \mathcal{F}, \mathbb{P})$ be a probability space and X be a metric space. If X_n, X: Ω → X is a sequence of random variables then X_n is said to converge weakly (or in distribution or in law) to X as n → ∞ if the sequence of pushforward measures (X_n)_∗(P) converges weakly to X_∗(P) in the sense of weak convergence of measures on X, as defined above.

References

• Ambrosio, L., Gigli, N. & Savaré, G. (2005). Gradient Flows in Metric Spaces and in the Space of Probability Measures. Basel: ETH Zürich, Birkhäuser Verlag. ISBN 3-7643-2428-7. 
• Billingsley, Patrick (1995). Probability and Measure. New York, NY: John Wiley & Sons, Inc.. ISBN 0-471-00710-2. 
• Billingsley, Patrick (1999). Convergence of Probability Measures. New York, NY: John Wiley & Sons, Inc.. ISBN 0-471-19745-9. 

See also

• Convergence of random variables
• Prokhorov's theorem
• Lévy–Prokhorov metric
• Tightness of measures