Moment-generating function

From Wikipedia, the free encyclopedia

(Redirected from Moment generating function)

In probability theory and statistics, the moment-generating function of a random variable X is

$M_X(t) := E\left(e^{tX}\right), \quad t \in \mathbb{R},$

wherever this expectation exists.

The moment-generating function is so called because, if it exists on an open interval around t = 0, then it is the ordinary generating function of the moments of the probability distribution:

$E \left( X^n \right) = M_X^{(n)}(0) = \frac{d^n M_X}{dt^n}(0).$

If the moment generating function is defined on such an interval, then it uniquely determines a probability distribution.

A key problem with moment-generating functions is that moments and the moment-generating function may not exist, as the integrals need not converge. By contrast, the characteristic function always exists (because the integral is a bounded function on a space of finite measure), and thus may be used instead.

More generally, where $\mathbf X = ( X_1, \ldots, X_n)$ , an n-dimensional random vector, one uses $\mathbf t \cdot \mathbf X = \mathbf t^\mathrm T\mathbf X$ instead of $t X$ :

$M_{\mathbf X}(\mathbf t) := E\left(e^{\mathbf t^\mathrm T\mathbf X}\right).$

[edit] Calculation

If X has a continuous probability density function f(x) then the moment generating function is given by

$M_X(t) = \int_{-\infty}^\infty e^{tx} f(x)\,\mathrm{d}x$

$= \int_{-\infty}^\infty \left( 1+ tx + \frac{t^2x^2}{2!} + \cdots\right) f(x)\,\mathrm{d}x$

$= 1 + tm_1 + \frac{t^2m_2}{2!} +\cdots,$

where $m i$ is the ith moment. $M X ( - t)$ is just the two-sided Laplace transform of f(x).

Regardless of whether the probability distribution is continuous or not, the moment-generating function is given by the Riemann-Stieltjes integral

$M_X(t) = \int_{-\infty}^\infty e^{tx}\,dF(x)$

where F is the cumulative distribution function.

If X₁, X₂, ..., X_n is a sequence of independent (and not necessarily identically distributed) random variables, and

$S_n = \sum_{i=1}^n a_i X_i,$

where the a_i are constants, then the probability density function for S_n is the convolution of the probability density functions of each of the X_i and the moment-generating function for S_n is given by

$M_{S_n}(t)=M_{X_1}(a_1t)M_{X_2}(a_2t)\cdots M_{X_n}(a_nt).$

For vector-valued random variables X with real components, the moment-generating function is given by

$M_X(t) = E\left( e^{\langle t, X \rangle}\right)$

where t is a vector and $\langle t, X \rangle$ is the dot product.

[edit] Relation to other functions

Related to the moment-generating function are a number of other transforms that are common in probability theory:

characteristic function: The characteristic function $\varphi_X(t)$ is related to the moment-generating function via $\varphi_X(t) = M_{iX}(t) = M_X(it):$ the characteristic function is the moment-generating function of iX or the moment generating function of X evaluated on the imaginary axis.

cumulant-generating function: The cumulant-generating function is defined as the logarithm of the moment-generating function; some instead define the cumulant-generating function as the logarithm of the characteristic function, while others call this latter the second cumulant-generating function.

probability-generating function: The probability-generating function is defined as $G(z) = E[z^X].\,$ This immediately implies that $G(e^t) = E[e^{tX}] = M_X(t).\,$