Confusion and diffusion

In cryptography, confusion and diffusion are two properties of the operation of a secure cipher identified by Claude Shannon in his 1945 classified report A Mathematical Theory of Cryptography. These properties, when present, work together to thwart the application of statistics and other methods of cryptanalysis.

Confusion in a symmetric cipher is obscuring the local correlation between the input (plaintext) and output (ciphertext) by varying the application of the key to the data, while diffusion is hiding the plaintext statistics by spreading it over a larger area of ciphertext. Although ciphers can be confusion-only (substitution cipher, one-time pad) or diffusion-only (transposition cipher), any "reasonable" block cipher uses both confusion and diffusion. These concepts are also important in the design of cryptographic hash functions and pseudorandom number generators, where decorrelation of the generated values is the main feature, diffusion (and its avalanche effect) is also applicable to non-cryptographic hash functions.

Definition

Confusion

Confusion means that each binary digit (bit) of the ciphertext should depend on several parts of the key, obscuring the connections between the two.

The property of confusion hides the relationship between the ciphertext and the key.

This property makes it difficult to find the key from the ciphertext and if a single bit in a key is changed, the calculation of most or all of the bits in the ciphertext will be affected.

Confusion increases the ambiguity of ciphertext and it is used by both block and stream ciphers.

In substitution–permutation networks, confusion is provided by substitution boxes.

Diffusion

Diffusion means that if we change a single bit of the plaintext, then about half of the bits in the ciphertext should change, and similarly, if we change one bit of the ciphertext, then about half of the plaintext bits should change. This is equivalent to the expectation that encryption schemes exhibit an avalanche effect.

The purpose of diffusion is to hide the statistical relationship between the ciphertext and the plain text. For example, diffusion ensures that any patterns in the plaintext, such as redundant bits, are not apparent in the ciphertext. Block ciphers achieve this by "diffusing" the information about the plaintext's structure across the rows and columns of the cipher.

In substitution–permutation networks, diffusion is provided by permutation boxes (a.k.a. permutation layer). In the beginning of the 21st century a consensus had appeared where the designers preferred the permutation layer to consist of linear Boolean functions, although nonlinear functions can be used, too.

Theory

In Shannon's original definitions, confusion refers to making the relationship between the ciphertext and the symmetric key as complex and involved as possible; diffusion refers to dissipating the statistical structure of plaintext over the bulk of ciphertext. This complexity is generally implemented through a well-defined and repeatable series of substitutions and permutations. Substitution refers to the replacement of certain components (usually bits) with other components, following certain rules. Permutation refers to manipulation of the order of bits according to some algorithm. To be effective, any non-uniformity of plaintext bits needs to be redistributed across much larger structures in the ciphertext, making that non-uniformity much harder to detect.

In particular, for a randomly chosen input, if one flips the i-th bit, then the probability that the j-th output bit will change should be one half, for any i and j—this is termed the strict avalanche criterion. More generally, one may require that flipping a fixed set of bits should change each output bit with probability one half.

One aim of confusion is to make it very hard to find the key even if one has a large number of plaintext-ciphertext pairs produced with the same key. Therefore, each bit of the ciphertext should depend on the entire key, and in different ways on different bits of the key. In particular, changing one bit of the key should change the ciphertext completely.

Practical applications

Design of a modern block cipher uses both confusion and diffusion, with confusion changing data between the input and the output by applying a key-dependent non-linear transformation (linear calculations are easier to reverse and thus are easier to break).

Confusion inevitably involves some diffusion, so a design with a very wide-input S-box can provide the necessary diffusion properties, but will be very costly in implementation. Therefore, the practical ciphers utilize relatively small S-boxes, operating on small groups of bits ("bundles"). For example, the design of AES has 8-bit S-boxes, Serpent − 4-bit, BaseKing and 3-way − 3-bit. Small S-boxes provide almost no diffusion, so the resources are spent on simpler diffusion transformations. For example, the wide trail strategy popularized by the Rijndael design, involves a linear mixing transformation that provides high diffusion, although the security proofs do not depend on the diffusion layer being linear.

One of the most researched cipher structures uses the substitution-permutation network (SPN) where each round includes a layer of local nonlinear permutations (S-boxes) for confusion and a linear diffusion transformation (usually a multiplication by a matrix over a finite field). Modern block ciphers mostly follow the confusion layer/diffusion layer model, with the efficiency of the diffusion layer estimated using the so-called branch number, a numerical parameter that can reach the value $s+1$ for s input bundles for the perfect diffusion transformation. Since the transformations that have high branch numbers (and thus require a lot of bundles as inputs) are costly in implementation, the diffusion layer is sometimes (for example, in the AES) composed from two sublayers, "local diffusion" that processes subsets of the bundles in a bricklayer fashion (each subset is transformed independently) and "dispersion" that makes the bits that were "close" (within one subset of bundles) to become "distant" (spread to different subsets and thus be locally diffused within these new subsets on the next round).

Analysis of AES

The Advanced Encryption Standard (AES) has both excellent confusion and diffusion. Its confusion look-up tables are very non-linear and good at destroying patterns. Its diffusion stage spreads every part of the input to every part of the output: changing one bit of input changes half the output bits on average. Both confusion and diffusion are repeated multiple times for each input to increase the amount of scrambling. The secret key is mixed in at every stage so that an attacker cannot precalculate what the cipher does.

None of this happens when a simple one-stage scramble is based on a key. Input patterns would flow straight through to the output. It might look random to the eye but analysis would find obvious patterns and the cipher could be broken.

See also

• Algorithmic information theory
• Avalanche effect
• Substitution–permutation network

Sources

• Claude E. Shannon, "A Mathematical Theory of Cryptography", Bell System Technical Memo MM 45-110-02, September 1, 1945.
• Claude E. Shannon, "Communication Theory of Secrecy Systems", Bell System Technical Journal, vol. 28–4, pages 656–715, 1949. [1] Archived 2007-06-05 at the Wayback Machine
• Wade Trappe and Lawrence C. Washington, Introduction to Cryptography with Coding Theory. Second edition. Pearson Prentice Hall, 2006.
• Li, Chaoyun; Wang, Qingju (2017). "Design of Lightweight Linear Diffusion Layers from Near-MDS Matrices" (PDF). IACR Transactions on Symmetric Cryptology. 1: 129–155. doi:10.13154/tosc.v2017.i1.129-155.
• Sajadieh, Mahdi; Dakhilalian, Mohammad; Mala, Hamid; Sepehrdad, Pouyan (2012). "Recursive Diffusion Layers for Block Ciphers and Hash Functions". Fast Software Encryption (PDF). Springer Berlin Heidelberg. pp. 385–401. doi:10.1007/978-3-642-34047-5_22. eISSN 1611-3349. ISSN 0302-9743.
• Daemen, Joan; Rijmen, Vincent (9 March 2013). The Design of Rijndael: AES - The Advanced Encryption Standard (PDF). Springer Science & Business Media. ISBN 978-3-662-04722-4. OCLC 1259405449.
• Stamp, Mark; Low, Richard M. (15 June 2007). Applied Cryptanalysis: Breaking Ciphers in the Real World. John Wiley & Sons. ISBN 978-0-470-14876-1. OCLC 1044324461.
• Liu, Yunwen; Rijmen, Vincent; Leander, Gregor (20 January 2018). "Nonlinear diffusion layers" (PDF). Designs, Codes and Cryptography. 86 (11): 2469–2484. doi:10.1007/s10623-018-0458-5. eISSN 1573-7586. ISSN 0925-1022.