Jump to content

crypt (C)

From Wikipedia, the free encyclopedia
(Redirected from Modular Crypt Format)
This article is about the Unix C library function. For the Unix utility, see Crypt (Unix).

crypt is a POSIX C library function. It is typically used to compute the hash of user account passwords. The function outputs a text string which also encodes the salt (usually the first two characters are the salt itself and the rest is the hashed result), and identifies the hash algorithm used (defaulting to the "traditional" one explained below). This output string forms a password record, which is usually stored in a text file.

More formally, crypt provides cryptographic key derivation functions for password validation and storage on Unix systems.

Relationship to Unix crypt utility

[edit]

There is an unrelated crypt utility in Unix, which is often confused with the C library function. To distinguish between the two, writers often refer to the utility program as crypt(1), because it is documented in section 1 of the Unix manual pages, and refer to the C library function as crypt(3), because its documentation is in manual section 3.[1]

Details

[edit]

This same crypt function is used both to generate a new hash for storage and also to hash a proffered password with a recorded salt for comparison.

Modern Unix implementations of the crypt library routine support a variety of hash schemes. The particular hash algorithm used can be identified by a unique code prefix in the resulting hashtext, following a de facto standard called Modular Crypt Format.[2][3][4]

The crypt() library function is also included in the Perl,[5] PHP,[6] Pike,[7] Python[8] (although it is now deprecated as of 3.11), and Ruby[9] programming languages.

Key derivation functions supported by crypt

[edit]

Over time various algorithms have been introduced. To enable backward compatibility, each scheme started using some convention of serializing the password hashes that was later called the Modular Crypt Format (MCF).[3] Old crypt(3) hashes generated before the de facto MCF standard may vary from scheme to scheme. A well-defined subset of the Modular Crypt Format was created during the Password Hashing Competition.[3] The format is defined as:[10]

$<id>[$<param>=<value>(,<param>=<value>)*][$<salt>[$<hash>]]

where

  • id: an identifier representing the hashing algorithm (such as 1 for MD5, 5 for SHA-256 etc.)
  • param name and its value: hash complexity parameters, like rounds/iterations count
  • salt: salt following the radix-64 alphabet (DES uses the decoded value)
  • hash: radix-64 encoded result of hashing the password and salt

The radix-64 encoding in crypt is called B64 and uses the alphabet ./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz which is different than the more common RFC 4648 base64

Scheme id Schema Example
DES Kyq4bCxAXJkbg
_ BSDi _EQ0.jzhSVeUyoSqLupI
1 md5crypt $1$etNnh7FA$OlM7eljE/B7F1J4XYNnk81
2, 2a, 2b, 2x, 2y bcrypt $2a$10$VIhIOofSMqgdGlL4wzE//e.77dAQGqntF/1dT7bqCrVtquInWy2qi
3 NTHASH $3$$8846f7eaee8fb117ad06bdd830b7586c
5 sha256crypt $5$9ks3nNEqv31FX.F$gdEoLFsCRsn/WRN3wxUnzfeZLoooVlzeF4WjLomTRFD
6 sha512crypt $6$qoE2letU$wWPRl.PVczjzeMVgjiA8LLy2nOyZbf7Amj3qLIL978o18gbMySdKZ7uepq9tmMQXxyTIrS12Pln.2Q/6Xscao0
7 scrypt $7$DU..../....2Q9obwLhin8qvQl6sisAO/$sHayJj/JBdcuD4lJ1AxiwCo9e5XSi8TcINcmyID12i8
8 (Cisco) PBKDF2 with SHA-256 $8$mTj4RZG8N9ZDOk$elY/asfm8kD3iDmkBe3hD2r4xcA/0oWS5V3os.O91u.
8 (JunOS) PBKDF2 $8$crypt-algo$hash-algo$iterations$salt$iv$tag$encrypted

$8$aes256-gcm$hmac-sha2-256$100$y/4YMC4YDLU$fzYDI4jjN6YCyQsYLsaf8A$Ilu4jLcZarD9YnyD /Hejww$okhBlc0cGakSqYxKww

gy gost-yescrypt $gy$jCT$HM87v.7RwpQLba8fDjNSk1$VgqS7k2OZWhFbAJVBye2vaA7ex/1VtU3a5fmL8Wv/26
md5 Solaris MD5 $md5,rounds=5000$GUBv0xjJ$$mSwgIswdjlTY0YxV7HBVm0
sha1 PBKDF1 with SHA-1 $sha1$40000$jtNX3nZ2$hBNaIXkt4wBI2o5rsi8KejSjNqIq
y yescrypt $y$j9T$F5Jx5fExrKuPp53xLKQ..1$X3DX6M94c7o.9agCG9G317fhZg9SqC.5i5rd.RhAtQ7

The PHC subset covers a majority of MCF hashes. A number of extra application-defined methods exist.[3]

Original implementation using the password as a key

[edit]

The original implementation of the crypt() library function[11] in Third Edition Unix[12] mimicked the M-209 cipher machine. Rather than encrypting the password with a key, which would have allowed the password to be recovered from the encrypted value and the key, it used the password itself as a key, and the password database contained the result of encrypting the password with this key.

Traditional DES-based scheme

[edit]

The original password encryption scheme was found to be too fast and thus subject to brute force enumeration of the most likely passwords.[11] In Seventh Edition Unix,[13] the scheme was changed to a modified form of the DES algorithm. A goal of this change was to make encryption slower. In addition, the algorithm incorporated a 12-bit salt in order to ensure that an attacker would be forced to crack each password independently as opposed to being able to target the entire password database simultaneously.

In detail, the user's password is truncated to eight characters, and those are coerced down to only 7-bits each; this forms the 56-bit DES key. That key is then used to encrypt an all-bits-zero block, and then the ciphertext is encrypted again with the same key, and so on for a total of 25 DES encryptions. A 12-bit salt is used to perturb the encryption algorithm, so standard DES implementations can't be used to implement crypt(). The salt and the final ciphertext are encoded into a printable string in a form of base64.

This is technically not encryption since the data (all bits zero) is not being kept secret; it's widely known to all in advance. However, one of the properties of DES is that it's very resistant to key recovery even in the face of known plaintext situations. It is theoretically possible that two different passwords could result in exactly the same hash. Thus the password is never "decrypted": it is merely used to compute a result, and the matching results are presumed to be proof that the passwords were "the same."

The advantages of this method have been that the hashtext can be stored and copied among Unix systems without exposing the corresponding plaintext password to the system administrators or other users. This portability has worked for over 30 years across many generations of computing architecture, and across many versions of Unix from many vendors.

Weaknesses of the traditional scheme

[edit]

The traditional DES-based crypt algorithm was originally chosen because DES was resistant to key recovery even in the face of "known plaintext" attacks, and because it was computationally expensive. On the earliest Unix machines it took over a full second to compute a password hash. This also made it reasonably resistant to dictionary attacks in that era. At that time password hashes were commonly stored in an account file (/etc/passwd) which was readable to anyone on the system. (This account file was also used to map user ID numbers into names, and user names into full names, etc.).

In the three decades since that time, computers have become vastly more powerful. Moore's Law has generally held true, so the computer speed and capacity available for a given financial investment has doubled over 20 times since Unix was first written. This has long since left the DES-based algorithm vulnerable to dictionary attacks, and Unix and Unix-like systems such as Linux have used "shadow" files for a long time, migrating just the password hash values out of the account file (/etc/passwd) and into a file (conventionally named /etc/shadow) which can only be read by privileged processes.

To increase the computational cost of password breaking, some Unix sites privately started increasing the number of encryption rounds on an ad hoc basis.[citation needed] This had the side effect of making their crypt() incompatible with the standard crypt(): the hashes had the same textual form, but were now calculated using a different algorithm. Some sites also took advantage of this incompatibility effect, by modifying the initial block from the standard all-bits-zero.[citation needed] This did not increase the cost of hashing, but meant that precomputed hash dictionaries based on the standard crypt() could not be applied.

BSDi extended DES-based scheme

[edit]

BSDi used a slight modification of the classic DES-based scheme. BSDi extended the salt to 24 bits and made the number of rounds variable (up to 224-1). The chosen number of rounds is encoded in the stored password hash, avoiding the incompatibility that occurred when sites modified the number of rounds used by the original scheme. These hashes are identified by starting with an underscore (_), which is followed by 4 characters representing the number of rounds then 4 characters for the salt.

The BSDi algorithm also supports longer passwords, using DES to fold the initial long password down to the eight 7-bit bytes supported by the original algorithm.

MD5-based scheme

[edit]

Poul-Henning Kamp designed a baroque and (at the time) computationally expensive algorithm based on the MD5 message digest algorithm. MD5 itself would provide good cryptographic strength for the password hash, but it is designed to be quite quick to calculate relative to the strength it provides. The crypt() scheme is designed to be expensive to calculate, to slow down dictionary attacks. The printable form of MD5 password hashes starts with $1$.

This scheme allows users to have any length password, and they can use any characters supported by their platform (not just 7-bit ASCII). (In practice many implementations limit the password length, but they generally support passwords far longer than any person would be willing to type.) The salt is also an arbitrary string, limited only by character set considerations.

First the passphrase and salt are hashed together, yielding an MD5 message digest. Then a new digest is constructed, hashing together the passphrase, the salt, and the first digest, all in a rather complex form. Then this digest is passed through a thousand iterations of a function which rehashes it together with the passphrase and salt in a manner that varies between rounds. The output of the last of these rounds is the resulting passphrase hash.

The fixed iteration count has caused this scheme to lose the computational expense that it once enjoyed and variable numbers of rounds are now favoured. In June 2012, Poul-Henning Kamp declared the algorithm insecure and encouraged users to migrate to stronger password scramblers.[14]

Blowfish-based scheme

[edit]
Main article: bcrypt

Niels Provos and David Mazières designed a crypt() scheme called bcrypt based on Blowfish, and presented it at USENIX in 1999.[15] The printable form of these hashes starts with $2$, $2a$, $2b$, $2x$ or $2y$ depending on which variant of the algorithm is used:

  • $2$ – Obsolete.
  • $2a$ – The current key used to identify this scheme. Since a major security flaw was discovered in 2011 in a non-OpenBSD crypt_blowfish implementation of the algorithm,[16] hashes indicated by this string are now ambiguous and might have been generated by the flawed implementation, or a subsequent fixed, implementation. The flaw may be triggered by some password strings containing non-ASCII (8th-bit-set) characters.
  • $2b$ – Used by recent OpenBSD implementations to include a mitigation to a wraparound problem.[17] Previous versions of the algorithm have a problem with long passwords. By design, long passwords are truncated at 72 characters, but there is a byte integer wraparound problem with certain password lengths resulting in weak hashes.[18]
  • $2x$ – A flag added after the crypt_blowfish bug discovery. Old hashes can be renamed to be $2x$ to indicate that they were generated with the broken algorithm. These hashes are still weak, but at least it's clear which algorithm was used to generate them.
  • $2y$ – A flag in crypt_blowfish to unambiguously use the new, corrected algorithm. On an older implementation suffering from the bug, $2y$ simply won't work. On a newer, fixed implementation, it will produce the same result as using $2b$.

Blowfish is notable among block ciphers for its expensive key setup phase. It starts off with subkeys in a standard state, then uses this state to perform a block encryption using part of the key, and uses the result of that encryption (really, a hashing) to replace some of the subkeys. Then it uses this modified state to encrypt another part of the key, and uses the result to replace more of the subkeys. It proceeds in this fashion, using a progressively modified state to hash the key and replace bits of state, until all subkeys have been set.

The number of rounds of keying is a power of two, which is an input to the algorithm. The number is encoded in the textual hash, e.g. $2y$10...

NT hash scheme

[edit]

FreeBSD implemented support for the NT LAN Manager hash algorithm to provide easier compatibility with NT accounts via MS-CHAP.[19] The NT-Hash algorithm is known to be weak, as it uses the deprecated md4 hash algorithm without any salting.[20] FreeBSD used the $3$ prefix for this. Its use is not recommended, as it is easily broken.[1]

SHA2-based scheme

[edit]

The commonly used MD5 based scheme has become easier to attack as computer power has increased. Although the Blowfish-based system has the option of adding rounds and thus remain a challenging password algorithm, it does not use a NIST-approved algorithm. In light of these facts, Ulrich Drepper [de] of Red Hat led an effort to create a scheme based on the SHA-2 (SHA-256 and SHA-512) hash functions.[21] The printable form of these hashes starts with $5$ (for SHA-256) or $6$ (for SHA-512) depending on which SHA variant is used. Its design is similar to the MD5-based crypt, with a few notable differences:[21]

  • It avoids adding constant data in a few steps.
  • The MD5 algorithm would repeatedly add the first letter of the password;[citation needed] this step was changed significantly.
  • Inspired by Sun's crypt() implementation, functionality to specify the number of iterations (rounds) the main loop in the algorithm performs was added[22][23]
  • The number of iterations is 5000 by default, with a minimum of 1000, and a maximum of 999,999,999.[24]

The specification and sample code have been released into the public domain; it is often referred to as "SHAcrypt".[24]

Other hashes

[edit]
$y$
yescrypt is an extension of scrypt ($7$) and a PHC finalist. It is used in several Linux distributions as an alternative to the existing schemes.[25] To use this hash, the libcrypt from glibc is replaced with a backward-compatible one from the "libxcrypt" project.[26]
$argon2d$, $argon2i$, $argon2ds$
These are PHC-assigned names for the Argon2 algorithm, but do not seem to be widely used.

Additional formats, if any, are described in the man pages of implementations.[27]

Archaic Unix schemes

[edit]

BigCrypt is the modified version of DES-Crypt used on HP-UX, Digital Unix, and OSF/1. The main difference between it and DES is that BigCrypt uses all the characters of a password, not just the first 8, and has a variable length hash.[28]

Crypt16 is the minor modification of DES, which allows passwords of up to 16 characters. Used on Ultrix and Tru64.[29]

Support in operating systems

[edit]
Scheme id Scheme Linux (glibc) Linux (libxcrypt)[30][31] Linux (musl) Linux (uClibc) FreeBSD NetBSD OpenBSD Solaris MacOS
DES Yes Yes Yes Yes Yes Yes Yes Yes Yes
_ BSDi No Yes No No Yes Yes Yes No Yes
1 MD5 Yes Yes Yes Yes Yes Yes Yes Yes No
2, 2a, 2b, 2x, 2y bcrypt No Yes Yes No Yes Yes Yes Yes No
3 NTHASH No Yes No No Yes No No No No
5 SHA-256 2.7+ Yes Yes Yes 8.3+ No No Yes No
6 SHA-512 2.7+ Yes Yes Yes 8.3+ No No Yes No
7 scrypt No Yes No No Yes No No Yes No
md5 Solaris MD5 No Yes No No No No No Yes No
sha1 PBKDF1 with SHA1 No Yes No No No Yes No No No
gy gost-yescrypt No Yes No No No No No No No
y yescrypt No Yes No No No No No No No

Linux

[edit]

The GNU C Library (glibc) used by almost all Linux distributions provides an implementation of the crypt function which supports the DES, MD5, and (since version 2.7) SHA-2 based hashing algorithms mentioned above. Ulrich Drepper, the glibc maintainer, rejected bcrypt (scheme 2) support since it isn't approved by NIST.[32] A public domain crypt_blowfish library is available for systems without bcrypt. It has been integrated into glibc in SUSE Linux.[33]

In August 2017, glibc announced plans to remove its crypt implementation completely. In response, a number of Linux distributions (including, but not limited to, Fedora and Debian) have switched to libxcrypt, an ABI-compatible implementation that additionally supports new algorithms, including bcrypt and yescrypt.[34]

The musl C library supports schemes 1, 2, 5, and 6, plus the tradition DES scheme. The traditional DES code is based on the BSD FreeSec, with modification to be compatible with the glibc UFC-Crypt.[35]

macOS

[edit]

Darwin's native crypt() provides limited functionality, supporting only DES and BSDi. OS X uses a few systems for its own password hashes, ranging from the old NeXTStep netinfo to the newer directory services (ds) system.[36][37]

See also

[edit]

References

[edit]
  1. ^ a b crypt(3) – FreeBSD Library Functions Manual
  2. ^ Simson Garfinkel, Alan Schwartz, Gene Spafford. "Practical Unix & Internet Security". 2003. section "4.3.2.3 crypt16( ), DES Extended, and Modular Crypt Format". "The Modular Crypt Format (MCF) specifies an extensible scheme for formatting encrypted passwords. MCF is one of the most popular formats for encrypted passwords"
  3. ^ a b c d "Modular Crypt Format — Passlib v1.7.1 Documentation". Pythonhosted.org. Retrieved 2 December 2018.
  4. ^ "ademarre/binary-mcf". GitHub.com. Retrieved 2 December 2018.
  5. ^ "crypt - perldoc.perl.org". Perldoc.perl.org. Retrieved 2 December 2018.
  6. ^ "PHP: crypt - Manual". Us.php.net. Retrieved 2 December 2018.
  7. ^ "crypt()". Archived from the original on 2012-10-02. Retrieved 2013-02-09.
  8. ^ "crypt — Function to check Unix passwords — Python 3.7.1 documentation". Docs.python.org. Retrieved 2 December 2018.
  9. ^ "Class: String (Ruby 2.5.3)". Ruby-doc.org. Retrieved 2 December 2018.
  10. ^ Password Hash Competition. "PHC string format". Github.
  11. ^ a b Morris, Robert; Thompson, Ken (1978-04-03). "Password Security: A Case History". Bell Laboratories. Retrieved 2013-12-17.
  12. ^ "crypt – password encoding". UNIX Third Edition Programmers' Manual. 1973-01-15.
  13. ^ "crypt, setkey, encrypt – DES encryption". UNIX Seventh Edition Programmers' Manual. 1979.
  14. ^ "Md5crypt Password scrambler is no longer considered safe by author — PHKs Bikeshed". Phk.freebsd.dk. Archived from the original on 17 March 2018. Retrieved 2 December 2018.
  15. ^ Provos, Niels; Mazières, David (1999). "A Future-Adaptable Password Scheme". Proceedings of 1999 USENIX Annual Technical Conference: 81–92.
  16. ^ Designer, Solar (2011-06-21). "crypt_blowfish 1.1; Owl glibc security update". See also CVE-2011-2483.
  17. ^ "src/lib/libc/crypt/bcrypt.c – view – 1.27". Cvsweb.openbsd.org. Retrieved 2016-05-14.
  18. ^ Designer, Solar (2012-01-02). "OpenBSD bcrypt 8-bit key_len wraparound".
  19. ^ "NT MD4 password hash as new password encryption method for FreeBSD". Mail-archive.com. Retrieved 2 December 2018.
  20. ^ "The NTLM Authentication Protocol and Security Support Provider". Davenport.sourceforge.net. Retrieved 2 December 2018.
  21. ^ a b Drepper, Ulrich (19 September 2007). "Unix crypt with SHA-256/512". Retrieved 21 November 2018.
  22. ^ Sun Microsystems. "crypt_sunmd5(5) man page". Archived from the original on 2008-04-16. Retrieved 2008-03-05.
  23. ^ Muffett, Alec (2005-12-05). "OpenSolaris, Pluggable Crypt, and the SunMD5 Password Hash Algorithm". Retrieved 2012-08-11.
  24. ^ a b Drepper, Ulrich. "Unix crypt using SHA-256 and SHA-512".
  25. ^ "FESCo Says "Yes" To Fedora 35 Using Yescrypt For Hashing Shadow Passwords - Phoronix". www.phoronix.com.
  26. ^ "Changes/yescrypt as default hashing method for shadow - Fedora Project Wiki". libxcrypt: Is already capable for computing yescrypt hashes since v4.3.
  27. ^
  28. ^ "passlib.hash.bigcrypt - BigCrypt — Passlib v1.7.1 Documentation". Pythonhosted.org. Retrieved 2 December 2018.
  29. ^ "passlib.hash.crypt16 - Crypt16 — Passlib v1.7.1 Documentation". Pythonhosted.org. Retrieved 2 December 2018.
  30. ^ libxcrypt - Extended crypt library for descrypt, md5crypt, bcrypt, and others
  31. ^ "Debian manpage crypt". manpages.debian.org. Retrieved 11 Jan 2022. – Debian bullseye uses libxcrypt. See https://tracker.debian.org/pkg/libxcrypt .
  32. ^ "bcrypt support for passwords in /etc/shadow - Red Hat Customer Portal". Access.redhat.com. 10 July 2018. Retrieved 2 December 2018.
  33. ^ "bcrypt password hashing ("password encryption") for your software and your servers". www.openwall.com.
  34. ^ "Changes/Replace glibc libcrypt with libxcrypt - Fedora Project Wiki". fedoraproject.org.
  35. ^ "crypt_r.c\crypt\src - musl - musl - an implementation of the standard library for Linux-based systems". git.musl-libc.org.
  36. ^ "How Mac OS X Implements Password Authentication". Dribin.org. 7 April 2006. Retrieved 2 December 2018.
  37. ^ "How to crack Mac OS X Passwords - Online Hash Crack". Onlinehashcrack.com. Retrieved 2 December 2018.
[edit]
Retrieved from "https://en.wikipedia.org/w/index.php?title=Crypt_(C)&oldid=1251819114#Key_derivation_functions_supported_by_crypt"