Chunked transfer encoding

Chunked Transfer Encoding is a mechanism that allows HTTP messages to be split in several parts. This can be applied to both HTTP requests (from client to server) and HTTP responses (from server to client). For example, let us consider the way in which an HTTP server may transmit data to a client application (usually a web browser). Normally, data delivered in HTTP responses is sent in one piece, whose length is indicated by the Content-Length header field. The length of the data is important, because the client needs to know where the response ends and any following response starts. With chunked encoding however, the data is broken up into a series of blocks of data and transmitted in one or more "chunks" so that a server may start sending data before it knows the final size of the content that it's sending. Often, the size of these blocks is the same, but this is not always the case.

HTTP servers sometimes use compression (gzip or deflate) to allow reducing time spent for transmission. Chunked transfer encoding can be used to delimit parts of the compressed object. In this case, it is worth noting that the chunks are not individually compressed. Instead, the complete payload is compressed and the output of the compression process is chunked using the scheme described in this article. In the case of compression, chunked encoding has the benefit that the compression can be performed on the fly while the data is delivered, as opposed to completing the compression process beforehand to determine the final size.

The chunked transfer mechanism is considered to be always acceptable, even if not listed in the TE request header, and when used with other transfer mechanisms, should always be applied last to the transferred data and never more than one time. This transfer coding method also allows additional entity headers to be sent after the last chunk if the client specified the "trailers" parameter as an argument of the TE header. The origin server of the response can also decide to send additional entity trailers even if the client did not specify the "trailers" option in the TE request header, but only if the metadata is optional (i.e. the client can use the received entity without them). Whenever the trailers are used, the server should list their names in the Trailer header field; 3 headers are specifically prohibited from appearing as a trailer field: Transfer-Encoding, Content-Length and Trailer.

Format

If a Transfer-Encoding header with a value of chunked is specified in an HTTP message (either a request sent by a client or the response from the server), the body of the message is made of an unspecified number of chunks ending with a last, zero-sized, chunk.

Each non-empty chunk starts with the number of octets of the data it embeds (size written in hexadecimal) followed by a CRLF (carriage return and line feed), and the data itself. The chunk is then closed with a CRLF. In some implementations, white space characters (0x20) are padded between chunk-size and the CRLF.

The last chunk is a single line, simply made of the chunk-size (0), some optional padding white spaces and the terminating CRLF. It is not followed by any data, but optional trailers can be sent using the same syntax as the message headers.

The message is finally closed by a final CRLF combination.

Sample

Encoded response

HTTP/1.1 200 OK
Content-Type: text/plain
Transfer-Encoding: chunked

23
This is the data in the first chunk

1C
and this is the second one

0

Encoded response (hex)

0000:0000   48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d   HTTP/1.1 200 OK.
0000:0010   0a 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 74   .Content-Type: t
0000:0020   65 78 74 2f 70 6c 61 69 6e 0d 0a 54 72 61 6e 73   ext/plain..Trans
0000:0030   66 65 72 2d 45 6e 63 6f 64 69 6e 67 3a 20 63 68   fer-Encoding: ch
0000:0040   75 6e 6b 65 64 0d 0a 0d 0a 32 33 0d 0a 54 68 69   unked....23..Thi
0000:0050   73 20 69 73 20 74 68 65 20 64 61 74 61 20 69 6e   s is the data in
0000:0060   20 74 68 65 20 66 69 72 73 74 20 63 68 75 6e 6b    the first chunk
0000:0070   0d 0a 0d 0a 31 43 0d 0a 61 6e 64 20 74 68 69 73   ....1C..and this
0000:0080   20 69 73 20 74 68 65 20 73 65 63 6f 6e 64 20 6f   is the second o
0000:0090   6e 65 0d 0a 30 0d 0a 0d 0a                        ne..0....

Decoded data

This is the data in the first chunk
and this is the second one

See RFC 2616 for further details; section 3.6.1 in particular.