# Markov kernel

In probability theory, a Markov kernel (also known as a stochastic kernel or probability kernel) is a map that plays the role, in the general theory of Markov processes, that the transition matrix does in the theory of Markov processes with a finite state space.[1]

## Formal definition

Let ${\displaystyle (X,{\mathcal {A}}),(Y,{\mathcal {B}})}$ be measurable spaces. A Markov kernel with source ${\displaystyle (X,{\mathcal {A}})}$ and target ${\displaystyle (Y,{\mathcal {B}})}$ is a map ${\displaystyle \kappa :X\times {\mathcal {B}}\to [0,1]}$ with the following properties:

1. The map ${\displaystyle x\mapsto \kappa (x,B)}$ is ${\displaystyle {\mathcal {A}}}$-measurable for every ${\displaystyle B\in {\mathcal {B}}.}$
2. The map ${\displaystyle B\mapsto \kappa (x,B)}$ is a probability measure on ${\displaystyle (Y,{\mathcal {B}})}$ for every ${\displaystyle x\in X}$.

In other words it associates to each point ${\displaystyle x\in X}$ a probability measure ${\displaystyle \kappa (x,\cdot )}$ on ${\displaystyle (Y,{\mathcal {B}})}$ such that, for every measurable set ${\displaystyle B\in {\mathcal {B}}}$, the map ${\displaystyle x\mapsto \kappa (x,B)}$ is measurable with respect to the ${\displaystyle \sigma }$-algebra ${\displaystyle {\mathcal {A}}.}$[2]

## Examples

### Simple random walk

Take ${\displaystyle X=Y=\mathbb {Z} ,{\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(\mathbb {Z} )}$ (the power set of ${\displaystyle \mathbb {Z} }$), then the Markov kernel ${\displaystyle \kappa }$ with

${\displaystyle \kappa (x,B)={\frac {1}{2}}\mathbf {1} _{B}(x-1)+{\frac {1}{2}}\mathbf {1} _{B}(x+1),\quad \forall x\in \mathbb {Z} ,\quad \forall B\in {\mathcal {P}}(\mathbb {Z} ),}$

where ${\displaystyle \mathbf {1} }$ is the indicator function, describes the transition rule for the random walk on ${\displaystyle \mathbb {Z} .}$

### Galton-Watson process

Take ${\displaystyle X=Y=\mathbb {N} ,{\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(\mathbb {N} ),}$ then

${\displaystyle \kappa (x,B)={\begin{cases}\mathbf {1} _{B}(0)&x=0\\\Pr(\xi _{1}+\cdots +\xi _{x}\in B)&x\neq 0\\\end{cases}}}$

with i.i.d. random variables ${\displaystyle \xi _{i}}$.

### General Markov processes with finite state space

Take ${\displaystyle X=Y,{\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(X)={\mathcal {P}}(Y)}$ and ${\displaystyle |X|=|Y|=n,}$ then the transition rule can be represented as a stochastic matrix ${\displaystyle (K_{ij})_{1\leq i,j\leq n}}$ with

${\displaystyle \forall i\in X:\qquad \sum _{j\in Y}K_{ij}=1.}$

In the convention of Markov kernels we write

${\displaystyle \kappa (i,B)=\sum _{j\in B}K_{ij},\qquad \forall i\in X,\quad \forall B\in {\mathcal {B}}}$.

### Construction of a Markov kernel

If ${\displaystyle \nu }$ is a finite measure on ${\displaystyle (Y,{\mathcal {B}})}$ and ${\displaystyle k:X\times Y\to \mathbb {R} _{+}}$ is a measurable function with respect to the product ${\displaystyle \sigma }$-algebra ${\displaystyle {\mathcal {A}}\otimes {\mathcal {B}}}$ and has the property

${\displaystyle \forall x\in X\qquad \int _{Y}k(x,y)\nu (\mathrm {d} y)=1,}$

then the mapping

${\displaystyle {\begin{cases}\kappa :X\times {\mathcal {B}}\to [0,1]\\\kappa (x,B)=\int _{B}k(x,y)\nu (\mathrm {d} y)\end{cases}}}$

defines a Markov kernel.[3]

## Properties

### Semidirect product

Let ${\displaystyle (X,{\mathcal {A}},P)}$ be a probability space and ${\displaystyle \kappa }$ a Markov kernel from ${\displaystyle (X,{\mathcal {A}})}$ to some ${\displaystyle (Y,{\mathcal {B}})}$. Then there exists a unique measure ${\displaystyle Q}$ on ${\displaystyle (X\times Y,{\mathcal {A}}\otimes {\mathcal {B}})}$, such that:

${\displaystyle Q(A\times B)=\int _{A}\kappa (x,B)dP(x),\quad \forall A\in {\mathcal {A}},\quad \forall B\in {\mathcal {B}}}$.

### Regular conditional distribution

Let ${\displaystyle (S,Y)}$ be a Borel space, ${\displaystyle X}$ a ${\displaystyle (S,Y)}$-valued random variable on the measure space ${\displaystyle (\Omega ,{\mathcal {F}},P)}$ and ${\displaystyle {\mathcal {G}}\subseteq {\mathcal {F}}}$ a sub-${\displaystyle \sigma }$-algebra. Then there exists a Markov kernel ${\displaystyle \kappa }$ from ${\displaystyle (\Omega ,{\mathcal {G}})}$ to ${\displaystyle (S,Y)}$, such that ${\displaystyle \kappa (\cdot ,B)}$ is a version of the conditional expectation ${\displaystyle \mathbb {E} [\mathbf {1} _{\{X\in B\}}|{\mathcal {G}}]}$ for every ${\displaystyle B\in Y}$, i.e.

${\displaystyle P(X\in B|{\mathcal {G}})=\mathbb {E} \left[\mathbf {1} _{\{X\in B\}}|{\mathcal {G}}\right]=\kappa (\omega ,B),\qquad P-a.s.\forall B\in {\mathcal {G}}.}$

It is called regular conditional distribution of ${\displaystyle X}$ given ${\displaystyle {\mathcal {G}}}$ and is not uniquely defined.

## References

1. ^ Reiss, R. D. (1993). "A Course on Point Processes". Springer Series in Statistics. ISBN 978-1-4613-9310-8. doi:10.1007/978-1-4613-9308-5.
2. ^ Klenke, Achim. Probability Theory: A Comprehensive Course (2 ed.). Springer. p. 180. doi:10.1007/978-1-4471-5361-0.
3. ^ Erhan, Cinlar (2011). Probability and Stochastics. New York: Springer. pp. 37–38. ISBN 978-0-387-87858-4.
§36. Kernels and semigroups of kernels