Markov kernel

In probability theory, a Markov kernel (or stochastic kernel) is a map that plays the role, in the general theory of Markov processes, that the transition matrix does in the theory of Markov processes with a finite state space.

Formal definition

Let $(X,{\mathcal {A}})$$(X,{\mathcal {A}})$, $(Y,{\mathcal {B}})$$(Y,{\mathcal {B}})$ be measurable spaces. A Markov kernel with source $(X,{\mathcal {A}})$$(X,{\mathcal {A}})$ and target $(Y,{\mathcal {B}})$$(Y,{\mathcal {B}})$ is a map $\kappa \colon X\times {\mathcal {B}}\to [0,1]$$\kappa \colon X\times {\mathcal {B}}\to [0,1]$ with the following properties:

1. The map $x\mapsto \kappa (x,B)$$x\mapsto \kappa (x,B)$ is ${\mathcal {A}}$${\mathcal {A}}$ - measureable for every $B\in {\mathcal {B}}$$B\in {\mathcal {B}}$.
2. The map $B\mapsto \kappa (x,B)$$B\mapsto \kappa (x,B)$ is a probability measure on $(Y,{\mathcal {B}})$$(Y,{\mathcal {B}})$ for every $x\in X$$x\in X$.

(i.e. It associates to each point $x\in X$$x\in X$ a probability measure $\kappa (x,.)$$\kappa (x,.)$ on $(Y,{\mathcal {B}})$$(Y,{\mathcal {B}})$ such that, for every measurable set $B\in {\mathcal {B}}$$B\in {\mathcal {B}}$, the map $x\mapsto \kappa (x,B)$$x\mapsto \kappa (x,B)$ is measurable with respect to the $\sigma$$\sigma$-algebra ${\mathcal {A}}$${\mathcal {A}}$.)

Examples

• Simple random walk: Take $X=Y=\mathbb {Z}$$X=Y=\mathbb {Z}$ and ${\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(\mathbb {Z} )$${\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(\mathbb {Z} )$, then the Markov kernel $\kappa$$\kappa$ with

$\kappa (x,B)={\frac {1}{2}}\mathbf {1} _{B}(x-1)+{\frac {1}{2}}\mathbf {1} _{B}(x+1),\quad \forall x\in \mathbb {Z} ,\quad \forall B\in {\mathcal {P}}(\mathbb {Z} )$$\kappa (x,B)={\frac {1}{2}}\mathbf {1} _{B}(x-1)+{\frac {1}{2}}\mathbf {1} _{B}(x+1),\quad \forall x\in \mathbb {Z} ,\quad \forall B\in {\mathcal {P}}(\mathbb {Z} )$,

describes the transition rule for the random walk on $\mathbb {Z}$$\mathbb {Z}$. Where $\mathbf {1}$$\mathbf {1}$ is the indicator function.

• Galton-Watson process: Take $X=Y=\mathbb {N}$$X=Y=\mathbb {N}$, ${\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(\mathbb {N} )$${\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(\mathbb {N} )$, then

$\kappa (x,B)={\begin{cases}\mathbf {1} _{B}(0)&\quad x=0,\\P[\xi _{1}+\dots +\xi _{x}\in B]&\quad {\text{else,}}\\\end{cases}}$$\kappa (x,B)={\begin{cases}\mathbf {1} _{B}(0)&\quad x=0,\\P[\xi _{1}+\dots +\xi _{x}\in B]&\quad {\text{else,}}\\\end{cases}}$

with i.i.d. random variables $\xi _{i}$$\xi _{i}$.

• General Markov processes with finite state space: Take $X=Y$$X=Y$, ${\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(X)={\mathcal {P}}(Y)$${\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(X)={\mathcal {P}}(Y)$ and $|X|=|Y|=n$$|X|=|Y|=n$, then the transition rule can be represented as a stochastic matrix $(K_{ij})_{1\leq i,j\leq n}$$(K_{ij})_{1\leq i,j\leq n}$ with $\Sigma _{j\in Y}K_{ij}=1$$\Sigma _{j\in Y}K_{ij}=1$ for every $i\in X$$i\in X$. In the convention of Markov kernels we write

$\kappa (i,B)=\Sigma _{j\in B}K_{ij},\quad \forall i\in X,\quad \forall B\in {\mathcal {B}}$$\kappa (i,B)=\Sigma _{j\in B}K_{ij},\quad \forall i\in X,\quad \forall B\in {\mathcal {B}}$.

• Construction of a Markov kernel: If $\nu$$\nu$ is a finite measure on $(Y,{\mathcal {B}})$$(Y,{\mathcal {B}})$ and $k:X\times Y\to \mathbb {R} _{+}$$k:X\times Y\to \mathbb {R} _{+}$ is a measurable function with respect to the product $\sigma$$\sigma$-algebra ${\mathcal {A}}\otimes {\mathcal {B}}$${\mathcal {A}}\otimes {\mathcal {B}}$ and has the property

$\int _{X}k(x,y)\nu (\mathrm {d} y)=1,$$\int _{X}k(x,y)\nu (\mathrm {d} y)=1,$

for all $x\in X$$x\in X$, then the mapping $\kappa :X\times {\mathcal {B}}\to [0,1]$$\kappa :X\times {\mathcal {B}}\to [0,1]$

$\kappa (x,B)=\int _{B}k(x,y)\nu (\mathrm {d} y),$$\kappa (x,B)=\int _{B}k(x,y)\nu (\mathrm {d} y),$

defines a Markov kernel.

Properties

Semidirect product

Let $(X,{\mathcal {A}},P)$$(X,{\mathcal {A}},P)$ be a probability space and $\kappa$$\kappa$ a Markov kernel from $(X,{\mathcal {A}})$$(X,{\mathcal {A}})$ to some $(Y,{\mathcal {B}})$$(Y,{\mathcal {B}})$.

Then there exists a unique measure $Q$$Q$ on $(X\times Y,{\mathcal {A}}\otimes {\mathcal {B}})$$(X\times Y,{\mathcal {A}}\otimes {\mathcal {B}})$, such that

$Q(A\times B)=\int _{A}\kappa (x,B)dP(x),\quad \forall A\in {\mathcal {A}},\quad \forall B\in {\mathcal {B}}$$Q(A\times B)=\int _{A}\kappa (x,B)dP(x),\quad \forall A\in {\mathcal {A}},\quad \forall B\in {\mathcal {B}}$.

Regular conditional distribution

Let $(S,Y)$$(S,Y)$ be a Borel space, $X$$X$ a $(S,Y)$$(S,Y)$ - valued random variable on the measure space $(\Omega ,{\mathcal {F}},P)$$(\Omega ,{\mathcal {F}},P)$ and ${\mathcal {G}}\subseteq {\mathcal {F}}$${\mathcal {G}}\subseteq {\mathcal {F}}$ a sub-$\sigma$$\sigma$-algebra.

Then there exists a Markov kernel $\kappa$$\kappa$ from $(\Omega ,{\mathcal {G}})$$(\Omega ,{\mathcal {G}})$ to $(S,Y)$$(S,Y)$, such that $\kappa (.,B)$$\kappa (.,B)$ is a version of the conditional expectation $E[\mathbf {1} _{\{X\in B\}}|{\mathcal {G}}]$$E[\mathbf {1} _{\{X\in B\}}|{\mathcal {G}}]$ for every $B\in Y$$B\in Y$, i.e.

$P[X\in B|{\mathcal {G}}]=E[\mathbf {1} _{\{X\in B\}}|{\mathcal {G}}]=\kappa (\omega ,B),\quad P-a.s.\forall B\in {\mathcal {G}}$$P[X\in B|{\mathcal {G}}]=E[\mathbf {1} _{\{X\in B\}}|{\mathcal {G}}]=\kappa (\omega ,B),\quad P-a.s.\forall B\in {\mathcal {G}}$.

It is called regular conditional distribution of $X$$X$ given ${\mathcal {G}}$${\mathcal {G}}$ and is not uniquely defined.

References

• Bauer, Heinz (1996), Probability Theory, de Gruyter, ISBN 3-11-013935-9 

§36. Kernels and semigroups of kernels