Khmaladze transformation

From Wikipedia, the free encyclopedia
Jump to: navigation, search

In statistics, the Khmaladze transformation is a mathematical tool used in constructing convenient goodness of fit tests for hypothetical distribution functions. More precisely, suppose X_1,\ldots, X_n are i.i.d., possibly multi-dimensional, random observations generated from an unknown probability distribution. A classical problem in statistics is to decide how well a given hypothetical distribution function F, or a given hypothetical parametric family of distribution functions  \{ F_\theta : \theta \in \Theta \}, fits the set of observations. The Khmaladze transformation allows us to construct goodness of fit tests with desirable properties. It is named after Estate V. Khmaladze.

Consider the sequence of empirical distribution functions F_n based on a sequence of i.i.d random variables, X_1,\ldots, X_n, as n increases. Suppose F is the hypothetical distribution function of each X_i. To test whether the choice of F is correct or not, statisticians use the normalized difference,

v_n(x)=\sqrt{n} [F_n(x)-F(x)].

This v_n, as a random process in x, is called the empirical process. Various functionals of v_n are used as test statistics. The change of the variable v_n(x)=u_n(t), t=F(x) transforms to the so-called uniform empirical process u_n. The latter is an empirical processes based on independent random variables U_i=F(X_i), which are uniformly distributed on [0,1] if the X_is do indeed have distribution function F.

This fact was discovered and first utilized by Kolmogorov (1933), Wald and Wolfowitz (1936) and Smirnov (1937) and, especially after Doob (1949) and Anderson and Darling (1952),[1] it led to the standard rule to choose test statistics based on v_n. That is, test statistics \psi(v_n,F) are defined (which possibly depend on the F being tested) in such a way that there exists another statistic \varphi(u_n) derived from the uniform empirical process, such that \psi(v_n,F)=\varphi(u_n). Examples are

\sup_x|v_n(x)|=\sup_t|u_n(t)|,\quad \sup_x\frac{|v_n(x)|}{a(F(x))}=\sup_t\frac {|u_n(t)|}{a(t)}


\int_{-\infty}^\infty v_n^2(x) \, dF(x)=\int_0^1 u_n^2(t)\,dt.

For all such functionals, their null distribution (under the hypothetical F) does not depend on F, and can be calculated once and then used to test any F.

However, it is only rarely that one needs to test a simple hypothesis, when a fixed F as a hypothesis is given. Much more often, one needs to verify parametric hypotheses where the hypothetical F=F_{\theta_n}, depends on some parameters \theta_n, which the hypothesis does not specify and which have to be estimated from the sample X_1,\ldots,X_n itself.

Although the estimators \hat \theta_n, most commonly converge to true value of \theta, it was discovered that the parametric,[2][3] or estimated, empirical process

\hat v_n(x)=\sqrt{n} [F_n(x)-F_{\hat\theta_n}(x)]

differs significantly from v_n and that the transformed process \hat u_n(t)=\hat v_n(x), t=F_{\hat\theta_n}(x) has a distribution for which the limit distribution, as n\to\infty, is dependent on the parametric form of F_{\theta} and on the particular estimator \hat\theta_n and, in general, within one parametric family, on the value of \theta.

From mid-1950s to the late-1980s, much work was done to clarify the situation and understand the nature of the process \hat v_n.

In 1981,[4] and then 1987 and 1993,[5] Khmaladze suggested to replace the parametric empirical process \hat v_n by its martingale part w_n only.

\hat v_n(x)-K_n(x)=w_n(x)

where K_n(x) is the compensator of \hat v_n(x). Then the following properties of w_n were established:

  • Although the form of K_n, and therefore, of w_n, depends on F_{\hat\theta_n}(x), as a function of both x and \theta_n, the limit distribution of the time transformed process
\omega_n(t)=w_n(x), t=F_{\hat \theta_n}(x)
is that of standard Brownian motion on [0,1], i.e., is again standard and independent of the choice of F_{\hat\theta_n}.
  • The relationship between \hat v_n and w_n and between their limits, is one to one, so that the statistical inference based on \hat v_n or on w_n are equivalent, and in w_n, nothing is lost compared to \hat v_n.
  • The construction of innovation martingale w_n could be carried over to the case of vector-valued X_1,\ldots,X_n, giving rise to the definition of the so-called scanning martingales in \mathbb R^d.

For a long time the transformation was, although known, still not used. Later, the work of researchers like Koenker, Stute, Bai, Koul, Koening, and others made it popular in econometrics and other fields of statistics.[citation needed]

See also[edit]


  1. ^ Anderson, T. W.; Darling, D. A. (1952). "Asymptotic Theory of Certain "Goodness of Fit" Criteria Based on Stochastic Processes". Annals of Mathematical Statistics 23 (2): 193–212. doi:10.1214/aoms/1177729437. 
  2. ^ Kac, M.; Kiefer, J.; Wolfowitz, J. (1955). "On Tests of Normality and Other Tests of Goodness of Fit Based on Distance Methods". Annals of Mathematical Statistics 26 (2): 189–211. doi:10.1214/aoms/1177728538. JSTOR 2236876. 
  3. ^ Gikhman (1954)[full citation needed]
  4. ^ Khmaladze, E. V. (1981). "Martingale Approach in the Theory of Goodness-of-fit Tests". Theor. Prob. Appl. 26 (2): 240–257. doi:10.1137/1126027. 
  5. ^ Khmaladze, E. V. (1993). "Goodness of fit Problems and Scanning Innovation Martingales". Annals of Statistics 21 (2): 798–829. doi:10.1214/aos/1176349152. JSTOR 2242262. 

Further reading[edit]

  • Koul, H. L.; Swordson, E. (2011). "Khmaladze transformation". International Encyclopedia of Statistical Science. Springer. pp. 715–718. doi:10.1007/978-3-642-04898-2_325.