Hoeffding's lemma

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search

In probability theory, Hoeffding's lemma is an inequality that bounds the moment-generating function of any bounded random variable.[1] It is named after the FinnishAmerican mathematical statistician Wassily Hoeffding.

The proof of Hoeffding's lemma uses Taylor's theorem and Jensen's inequality. Hoeffding's lemma is itself used in the proof of McDiarmid's inequality.

Statement of the lemma[edit]

Let X be any real-valued random variable such that almost surely, i.e. with probability one. Then, for all ,

or equivalently,

Proof[edit]

Without loss of generality, by replacing by , we can assume , so that .

Since is a convex function of , we have that for all ,

So,

where . By computing derivatives, we can conclude

and for all .

From Taylor's theorem, for some

Hence, .

See also[edit]

Notes[edit]

  1. ^ Pascal Massart (26 April 2007). Concentration Inequalities and Model Selection: Ecole d'Eté de Probabilités de Saint-Flour XXXIII - 2003. Springer. p. 21. ISBN 978-3-540-48503-2.