Markov kernel

(Redirected from Stochastic kernel)

In probability theory, a Markov kernel (also known as a stochastic kernel or probability kernel) is a map that plays the role, in the general theory of Markov processes, that the transition matrix does in the theory of Markov processes with a finite state space.[1]

Formal definition

Let ${\displaystyle (X,{\mathcal {A}})}$, ${\displaystyle (Y,{\mathcal {B}})}$ be measurable spaces. A Markov kernel with source ${\displaystyle (X,{\mathcal {A}})}$ and target ${\displaystyle (Y,{\mathcal {B}})}$ is a map ${\displaystyle \kappa \colon X\times {\mathcal {B}}\to [0,1]}$ with the following properties:

1. The map ${\displaystyle x\mapsto \kappa (x,B)}$ is ${\displaystyle {\mathcal {A}}}$ - measurable for every ${\displaystyle B\in {\mathcal {B}}}$.
2. The map ${\displaystyle B\mapsto \kappa (x,B)}$ is a probability measure on ${\displaystyle (Y,{\mathcal {B}})}$ for every ${\displaystyle x\in X}$.

(i.e. It associates to each point ${\displaystyle x\in X}$ a probability measure ${\displaystyle \kappa (x,.)}$ on ${\displaystyle (Y,{\mathcal {B}})}$ such that, for every measurable set ${\displaystyle B\in {\mathcal {B}}}$, the map ${\displaystyle x\mapsto \kappa (x,B)}$ is measurable with respect to the ${\displaystyle \sigma }$-algebra ${\displaystyle {\mathcal {A}}}$.)[2]

Examples

• Simple random walk: Take ${\displaystyle X=Y=\mathbb {Z} }$ and ${\displaystyle {\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(\mathbb {Z} )}$ (the power set of ${\displaystyle \mathbb {Z} }$), then the Markov kernel ${\displaystyle \kappa }$ with
${\displaystyle \kappa (x,B)={\frac {1}{2}}\mathbf {1} _{B}(x-1)+{\frac {1}{2}}\mathbf {1} _{B}(x+1),\quad \forall x\in \mathbb {Z} ,\quad \forall B\in {\mathcal {P}}(\mathbb {Z} )}$,

describes the transition rule for the random walk on ${\displaystyle \mathbb {Z} }$, where ${\displaystyle \mathbf {1} }$ is the indicator function.

• Galton-Watson process: Take ${\displaystyle X=Y=\mathbb {N} }$, ${\displaystyle {\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(\mathbb {N} )}$, then
${\displaystyle \kappa (x,B)={\begin{cases}\mathbf {1} _{B}(0)&\quad x=0,\\P[\xi _{1}+\dots +\xi _{x}\in B]&\quad {\text{else,}}\\\end{cases}}}$

with i.i.d. random variables ${\displaystyle \xi _{i}}$.

• General Markov processes with finite state space: Take ${\displaystyle X=Y}$, ${\displaystyle {\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(X)={\mathcal {P}}(Y)}$ and ${\displaystyle |X|=|Y|=n}$, then the transition rule can be represented as a stochastic matrix ${\displaystyle (K_{ij})_{1\leq i,j\leq n}}$ with ${\displaystyle \Sigma _{j\in Y}K_{ij}=1}$ for every ${\displaystyle i\in X}$. In the convention of Markov kernels we write
${\displaystyle \kappa (i,B)=\Sigma _{j\in B}K_{ij},\quad \forall i\in X,\quad \forall B\in {\mathcal {B}}}$.
${\displaystyle \int _{Y}k(x,y)\nu (\mathrm {d} y)=1,}$

for all ${\displaystyle x\in X}$, then the mapping ${\displaystyle \kappa :X\times {\mathcal {B}}\to [0,1]}$

${\displaystyle \kappa (x,B)=\int _{B}k(x,y)\nu (\mathrm {d} y),}$

defines a Markov kernel.[3]

Properties

Semidirect product

Let ${\displaystyle (X,{\mathcal {A}},P)}$ be a probability space and ${\displaystyle \kappa }$ a Markov kernel from ${\displaystyle (X,{\mathcal {A}})}$ to some ${\displaystyle (Y,{\mathcal {B}})}$.

Then there exists a unique measure ${\displaystyle Q}$ on ${\displaystyle (X\times Y,{\mathcal {A}}\otimes {\mathcal {B}})}$, such that

${\displaystyle Q(A\times B)=\int _{A}\kappa (x,B)dP(x),\quad \forall A\in {\mathcal {A}},\quad \forall B\in {\mathcal {B}}}$.

Regular conditional distribution

Let ${\displaystyle (S,Y)}$ be a Borel space, ${\displaystyle X}$ a ${\displaystyle (S,Y)}$ - valued random variable on the measure space ${\displaystyle (\Omega ,{\mathcal {F}},P)}$ and ${\displaystyle {\mathcal {G}}\subseteq {\mathcal {F}}}$ a sub-${\displaystyle \sigma }$-algebra.

Then there exists a Markov kernel ${\displaystyle \kappa }$ from ${\displaystyle (\Omega ,{\mathcal {G}})}$ to ${\displaystyle (S,Y)}$, such that ${\displaystyle \kappa (.,B)}$ is a version of the conditional expectation ${\displaystyle E[\mathbf {1} _{\{X\in B\}}|{\mathcal {G}}]}$ for every ${\displaystyle B\in Y}$, i.e.

${\displaystyle P[X\in B|{\mathcal {G}}]=E[\mathbf {1} _{\{X\in B\}}|{\mathcal {G}}]=\kappa (\omega ,B),\quad P-a.s.\forall B\in {\mathcal {G}}}$.

It is called regular conditional distribution of ${\displaystyle X}$ given ${\displaystyle {\mathcal {G}}}$ and is not uniquely defined.

References

1. ^ Reiss, R. D. (1993). "A Course on Point Processes". Springer Series in Statistics. ISBN 978-1-4613-9310-8. doi:10.1007/978-1-4613-9308-5.
2. ^ Klenke, Achim. Probability Theory: A Comprehensive Course (2 ed.). Springer. p. 180. doi:10.1007/978-1-4471-5361-0.
3. ^ Erhan, Cinlar (2011). Probability and Stochastics. New York: Springer. pp. 37–38. ISBN 978-0-387-87858-4.
§36. Kernels and semigroups of kernels