Jump to content

Markov chains on a measurable state space

From Wikipedia, the free encyclopedia

A Markov chain on a measurable state space is a discrete-time-homogeneous Markov chain with a measurable space as state space.

History

[edit]

The definition of Markov chains has evolved during the 20th century. In 1953 the term Markov chain was used for stochastic processes with discrete or continuous index set, living on a countable or finite state space, see Doob.[1] or Chung.[2] Since the late 20th century it became more popular to consider a Markov chain as a stochastic process with discrete index set, living on a measurable state space.[3][4][5]

Definition

[edit]

Denote with a measurable space and with a Markov kernel with source and target . A stochastic process on is called a time homogeneous Markov chain with Markov kernel and start distribution if

is satisfied for any . One can construct for any Markov kernel and any probability measure an associated Markov chain.[4]

Remark about Markov kernel integration

[edit]

For any measure we denote for -integrable function the Lebesgue integral as . For the measure defined by we used the following notation:

Basic properties

[edit]

Starting in a single point

[edit]

If is a Dirac measure in , we denote for a Markov kernel with starting distribution the associated Markov chain as on and the expectation value

for a -integrable function . By definition, we have then .

We have for any measurable function the following relation:[4]

Family of Markov kernels

[edit]

For a Markov kernel with starting distribution one can introduce a family of Markov kernels by

for and . For the associated Markov chain according to and one obtains

.

Stationary measure

[edit]

A probability measure is called stationary measure of a Markov kernel if

holds for any . If on denotes the Markov chain according to a Markov kernel with stationary measure , and the distribution of is , then all have the same probability distribution, namely:

for any .

Reversibility

[edit]

A Markov kernel is called reversible according to a probability measure if

holds for any . Replacing shows that if is reversible according to , then must be a stationary measure of .

See also

[edit]

References

[edit]
  1. ^ Joseph L. Doob: Stochastic Processes. New York: John Wiley & Sons, 1953.
  2. ^ Kai L. Chung: Markov Chains with Stationary Transition Probabilities. Second edition. Berlin: Springer-Verlag, 1974.
  3. ^ Sean Meyn and Richard L. Tweedie: Markov Chains and Stochastic Stability. 2nd edition, 2009.
  4. ^ a b c Daniel Revuz: Markov Chains. 2nd edition, 1984.
  5. ^ Rick Durrett: Probability: Theory and Examples. Fourth edition, 2005.