# Total variation distance of probability measures

(Redirected from Total variation distance)

In probability theory, the total variation distance is a distance measure for probability distributions. It is an example of a statistical distance metric, and is sometimes just called "the" statistical distance.

## Definition

The total variation distance between two probability measures P and Q on a sigma-algebra $\mathcal{F}$ of subsets of the sample space $\Omega$ is defined via[1]

$\delta(P,Q)=\sup_{ A\in \mathcal{F}}\left|P(A)-Q(A)\right|.$

Informally, this is the largest possible difference between the probabilities that the two probability distributions can assign to the same event.

For a finite alphabet we can relate the total variation distance to the 1-norm of the difference of the two probability distributions as follows:[2]

$\delta(P,Q) = \frac 1 2 \|P-Q\|_1 = \frac 1 2 \sum_x \left| P(x) - Q(x) \right|\;.$

For arbitrary sample spaces, an equivalent definition of the total variation distance is

$\delta(P,Q) = \frac 1 2 \int_\Omega \left| f_P - f_Q \right|d\mu\;.$

where $\mu$ is an arbitrary positive measure such that both $P$ and $Q$ are absolutely continuous with respect to it and where $f_P$ and $f_Q$ are the Radon-Nikodym derivatives of $P$ and $Q$ with respect to $\mu$.

The total variation distance is related to the Kullback–Leibler divergence by Pinsker's inequality.