Markov kernel

(Redirected from Stochastic kernel)

In probability theory, a Markov kernel (or stochastic kernel) is a map that plays the role, in the general theory of Markov processes, that the transition matrix does in the theory of Markov processes with a finite state space.[1][2]

Formal definition

Let $(X,\mathcal A)$, $(Y,\mathcal B)$ be measurable spaces. A Markov kernel with source $(X,\mathcal A)$ and target $(Y,\mathcal B)$ is a map $\kappa \colon X \times \mathcal B \to [0,1]$ with the following properties:

1. The map $x \mapsto \kappa(x,B)$ is $\mathcal A$ - measureable for every $B \in \mathcal B$.
2. The map $B \mapsto \kappa(x,B)$ is a probability measure on $(Y, \mathcal B)$ for every $x \in X$.

(i.e. It associates to each point $x \in X$ a probability measure $\kappa(x,.)$ on $(Y,\mathcal B)$ such that, for every measurable set $B\in\mathcal B$, the map $x\mapsto \kappa(x,B)$ is measurable with respect to the $\sigma$-algebra $\mathcal A$.)

Examples

• Simple random walk: Take $X=Y=\Z$ and $\mathcal A = \mathcal B = \mathcal P(\Z)$, then the Markov kernel $\kappa$ with
$\kappa(x,B)=\frac{1}{2}\delta_{x-1}(B)+\frac{1}{2}\delta_{x+1}(B), \quad \forall x \in \Z \forall B \in \mathcal P(\Z)$,

describes the transition rule for the random walk on $\Z$.

• Galton-Watson process: Take $S=Y=\N$, $\mathcal A = \mathcal B = \mathcal P(\N)$, then
$\kappa(x,B)=\begin{cases} \delta_0(B) & \quad x=0,\\ P[\xi_1 + \dots + \xi_x \in B] & \quad \text{else,}\\ \end{cases}$

with i.i.d. random variables $\xi_i$.

• General Markov processes with finite state space: Take $X=Y$, $\mathcal A = \mathcal B = \mathcal P(X) = \mathcal P(Y)$ and $|X|=|Y|=n$, then the transition rule can be represented as a stochastic matrix $(K_{ij})_{1 \leq i,j \leq n}$ with $\Sigma_{j \in Y}K_{ij}=1$ for every $i \in X$. In the convention of Markov kernels we write
$\kappa(i,B)=\Sigma_{j \in B}K_{ij} \quad \forall i \in X \forall B \in \mathcal B$.

Properties

Semidirect product

Let $(X, \mathcal A, P)$ be a probability space and $\kappa$ a Markov kernel from $(X, \mathcal A)$ to some $(Y, \mathcal B)$. Then there exists a unique measure $Q$ on $(X \times Y, \mathcal A \otimes \mathcal B)$, s.t.

$Q(A \times B) = \int_A \kappa(x,B)dP(x), \quad \forall A \in \mathcal A \forall B \in \mathcal B$.

Regular conditional distribution

Let $(S,Y)$ be a Borel space, $X$ a $(S,Y)$ - valued random variable on the measure space $(\Omega, \mathcal F,P)$ and $\mathcal G \subseteq \mathcal F$ a sub-$\sigma$-algebra. Then there exists a Markov kernel $\kappa$ from $(\Omega, \mathcal G)$ to $(S,Y)$, s.t. $\kappa(.,B)$ is a version of the conditional expectation $E[\mathbf 1_{\{X \in B\}}| \mathcal G]$ for every $B \in Y$, i.e.

$P[X \in B|\mathcal G]=E[\mathbf 1_{\{X \in B\}}|\mathcal G]=\kappa(\omega,B), \quad P-a.s. \forall B \in \mathcal G$.

It is called regular conditional distribution of $X$ given $\mathcal G$ and is not uniquely defined.

References

1. ^ Epstein, P.; Howlett, P.; Schulze, M. S. (2003). "Distribution dynamics: Stratification, polarization, and convergence among OECD economies, 1870–1992". Explorations in Economic History 40: 78. doi:10.1016/S0014-4983(02)00023-2. edit
2. ^ Reiss, R. D. (1993). "A Course on Point Processes". Springer Series in Statistics. doi:10.1007/978-1-4613-9308-5. ISBN 978-1-4613-9310-8. edit
§36. Kernels and semigroups of kernels