Data processing inequality

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search

The Data processing inequality is an information theoretic concept which states that the information content of a signal cannot be increased via a local physical operation. This can be expressed concisely as 'post-processing cannot increase information'.[1]

Definition[edit]

Let three random variables form the Markov chain , implying that the conditional distribution of depends only on and is conditionally independent of . Specifically, we have such a Markov chain if the joint probability mass function can be written as

In this setting, no processing of Y , deterministic or random, can increase the information that Y contains about X. Using the mutual information, this can be written as :

With the equality if and only if , i.e. and contain the same information about , and also forms a Markov chain.[2]

See also[edit]

References[edit]

  1. ^ Beaudry, Normand (2012), "An intuitive proof of the data processing inequality", Quantum Information & Computation, 12 (5–6): 432–441, arXiv:1107.0740, Bibcode:2011arXiv1107.0740B
  2. ^ Cover; Thomas (2012). Elements of information theory. John Wiley & Sons.

External links[edit]