zlib

zlib

Stable release	1.2.6 / January 29, 2012 (2012-01-29)
Written in	C
Operating system	Cross-platform
Type	Data compression
License	zlib license
Website	http://zlib.net/

zlib is a software library used for data compression. zlib was written by Jean-Loup Gailly and Mark Adler and is an abstraction of the DEFLATE compression algorithm used in their gzip file compression program. zlib is also a crucial component of many software platforms including Linux, Mac OS X, and the iOS. It has also been used in gaming consoles such as the Playstation 3, Wii, and Xbox 360.

The first public version of zlib, 0.9, was released on 1 May 1995 and was originally intended for use with the libpng image library. It is free software, distributed under the zlib license.

Capabilities

Encapsulation

zlib compressed data is typically written with a gzip or a zlib wrapper. The wrapper encapsulates the raw DEFLATE data by adding a header and trailer. This provides stream identification and error detection that are not provided by the raw DEFLATE data.

The gzip header is larger than the zlib header, as it stores a file name and other file system information. This is the header format used in the ubiquitous gzip file format.

Algorithm

As of February 2010 zlib only supports one algorithm called DEFLATE that is a variation of LZ77 (Lempel–Ziv 1977). This algorithm provides good compression on a wide variety of data with minimal use of system resources. This is also the algorithm used in the ZIP archive format.

It is unlikely that the zlib format will ever be extended to use any other algorithms, though the header makes allowance for this possibility.

Resource use

The library provides facilities for control of processor and memory use. A compression level value may be supplied that trades-off speed with compression. There are also facilities for conserving memory. These are probably only useful in restricted memory environments such as some embedded systems.

Strategy

The compression can be optimized for specific types of data. If you are using the library to always compress specific types of data, then using a specific strategy may improve compression and performance. For example, if your data contains long lengths of repeated bytes, the RLE (run-length encoding) strategy may give good results at higher speed. For general data, the default strategy is preferred.

Error handling

Errors may be detected and skipped. Data corruption can be detected (as long as data is written with a zlib or gzip header - see above). Further, if full-flush points are written to the compressed stream, corrupt data can be skipped and the decompression will resynchronise at the next flush point. (No error recovery of the corrupt data is provided.) Full-flush points are useful for large data streams on unreliable channels, where some last data loss is unimportant (e.g. multimedia). However, creating too many flush points can dramatically affect speed and compression.

Data length

There is no limit to the length of data that can be compressed or decompressed. Repeated calls to the library allow an unlimited numbers of blocks of data to be handled. Some ancillary code (counters) may suffer from overflow for long data streams, but this does not affect the actual compression or decompression.

When compressing a long (or infinite) data stream, it would be advisable to write regular full-flush points.

Applications

Today, zlib is something of a de facto standard to the point that zlib and DEFLATE are often used interchangeably in standards documents. Thousands of applications rely on it for compression, directly or indirectly, including:

• The Linux kernel, where it is used to implement compressed network protocols, compressed file systems and to decompress the kernel image itself at boot time.
• libpng, the reference implementation for the PNG image format, which specifies DEFLATE as the stream compression for its bitmap data.
• Libwww, an API for Web applications like Web browser.
• The Apache HTTP server, which uses zlib to implement HTTP/1.1.
• The OpenSSH client and server, which rely on zlib to perform the optional compression offered by the Secure Shell protocol.
• The OpenSSL and GnuTLS security libraries, which can optionally use zlib to compress TLS connections.
• The FFmpeg multimedia library, which uses zlib to read and write the DEFLATE-compressed parts of stream formats such as Matroska.
• The rsync remote file synchronizer, which uses zlib to implement optional protocol compression.
• The dpkg and RPM package managers, which use zlib to unpack files from compressed software packages.
• The Subversion and CVS version control systems, which use zlib to compress traffic to and from remote repositories.
• The Git version control system that uses zlib to store the contents of its data objects (blobs, trees, commits and tags).
• The PostgreSQL RDBMS that uses zlib with custom dump format (pg_dump -Fc) for database backups.

zlib is also used in many embedded devices such as the Apple Inc. iPhone and Sony Playstation 3, because the code is portable, liberally-licensed, and has a relatively small memory footprint.

See also

• DEFLATE
• gzip
• LZ77
• ZIP (file format)

External links

• Official website
• RFC 1950—ZLIB Compressed Data Format
• RFC 1951—DEFLATE Compressed Data Format
• RFC 1952—GZIP file format