Tone mapping

This article is about technique to map colors of HDR images. For a general technique to map colors between images, see Color mapping.

Tone mapping is a technique used in image processing and computer graphics to map one set of colors to another in order to approximate the appearance of high dynamic range images in a medium that has a more limited dynamic range. Print-outs, CRT or LCD monitors, and projectors all have a limited dynamic range that is inadequate to reproduce the full range of light intensities present in natural scenes. Essentially, tone mapping addresses the problem of strong contrast reduction from the scene values (radiance) to the displayable range while preserving the image details and color appearance important to appreciate the original scene content.

Purpose and methods

The goals of tone mapping can be differently stated depending on the particular application. In some cases producing just aesthetically pleasing images is the main goal, while other applications might emphasize reproducing as many image details as possible, or maximizing the image contrast. The goal in realistic rendering applications might be to obtain a perceptual match between a real scene and a displayed image even though the display device is not able to reproduce the full range of luminance values.

Various tone mapping operators have been developed in the recent years . They all can be divided in two main types:

• global (or spatially uniform) operators: they are non-linear functions based on the luminance and other global variables of the image. Once the optimal function has been estimated according to the particular image, every pixel in the image is mapped in the same way, independent of the value of surrounding pixels in the image. Those techniques are simple and fast (since they can be implemented using look-up-tables), but they can cause a loss of contrast. Examples of common global tone mapping methods are contrast reduction and negative.

• local (or spatially varying) operators: the parameters of the non-linear function change in each pixel, according to features extracted from the surrounding parameters. In other words, the effect of the algorithm changes in each pixel according to the local features of the image. Those algorithms are more complicated than the global ones, they can show artifacts (e.g. halo effect and ringing), the output can look un-realistic, but they can provide the best performance, since the human vision is mainly sensitive to local contrast.

A simple example of global tone mapping filter is $L=\frac{Y}{Y+1}$. This function will map scene radiance values Y in the domain $[0,\infty)$ to a displayable output range of $[0,1)$. Global tone mapping can be done very fast and in a very flexible way.

A more sophisticated group of tone mapping algorithms is based on contrast or gradient domain methods, which are 'local'. Such operators concentrate on preserving contrast between neighboring regions rather than absolute value, an approach motivated by the fact that the human perception is most sensitive to contrast in images rather than absolute intensities. Those tone mapping methods usually produce very sharp images, which preserve very well small contrast details; however, this is often done at the cost of flattening an overall image contrast. Examples of such tone mapping methods include: gradient domain high dynamic range compression and A Perceptual Framework for Contrast Processing of High Dynamic Range Images (a tone mapping is one of the applications of this framework).

An interesting approach to tone mapping of HDR images is inspired by the anchoring theory of lightness perception . This theory explains many characteristics of the human visual system such as lightness constancy and its failures (as in the checker shadow illusion), which are important in the perception of images. The key concept of this tone mapping method (Lightness Perception in Tone Reproduction) is a decomposition of an HDR image into areas (frameworks) of consistent illumination and the local calculation of the lightness values. The net lightness of an image is calculated by merging of the frameworks proportionally to their strength. Particularly important is the anchoring—relating the luminance values to a known brightness value, namely estimating which luminance value is perceived as white in the scene. This approach to tone mapping does not affect the local contrast and preserves the natural colors of an HDR image due to the linear handling of luminance.

One simple form of tone mapping takes a standard image (not HDR – the dynamic range already compressed) and applies unsharp masking with a large radius, which increases local contrast rather than sharpening. See unsharp masking: local contrast enhancement for details.

Tone mapping in digital photography

Tone mapped HDR image of Dundas Square; Tone mapping was done as post-processing technique, using Photomatix photographic software.

Forms of tone mapping long precede digital photography. The manipulation of film and development process to render high contrast scenes, especially those shot in bright sunlight, on printing paper with a relatively low dynamic range, is effectively a form of tone mapping, although it is not usually called that. Local adjustment of tonality in film processing is primarily done via dodging and burning, and is particularly advocated by and associated with Ansel Adams, as described in his book The Print; see also his Zone System.

The normal process of exposure compensation, brightening shadows and altering contrast applied globally to digital images as part of a professional or serious amateur workflow is also a form of tone mapping.

However, HDR tone mapping, usually using local operators, has become increasingly popular amongst digital photographers as a post-processing technique, where several exposures at different shutter speeds are combined to produce an HDR image and a tone mapping operator is then applied to the result. There are now many examples of locally tone mapped digital images, inaccurately known as "HDR photographs", on the internet, and these are of varying quality. This popularity is partly driven by the distinctive appearance of locally tone mapped images, which many people find attractive, and partly by a desire to capture high-contrast scenes that are hard or impossible to photograph in a single exposure, and may not render attractively even when they can be captured. Although digital sensors actually capture a higher dynamic range than film, they completely lose detail in extreme highlights, clipping them to pure white, producing an unattractive result when compared with negative film, which tends to retain colour and some detail in highlights.

In some cases local tone mapping is used even though the dynamic range of the source image could be captured on the target media, either to produce the distinctive appearance of a locally tone mapped image, or to produce an image closer to the photographer's artistic vision of the scene by removing sharp contrasts, which often look unattractive. In some cases, tone mapped images are produced from a single exposure which is then manipulated with conventional processing tools to produce the inputs to the HDR image generation process. This avoids the artifacts that can appear when different exposures are combined, due to moving objects in the scene or camera shake. However, when tone mapping is applied to a single exposure in this way, the intermediate image has only normal dynamic range, and the amount of shadow or highlight detail that can be rendered is only that which was captured in the original exposure.

Example of the imaging process

Tone mapped High dynamic range image example showing stained glass windows in south alcove of Old Saint Paul's, Wellington, New Zealand.

The six individual exposures used to create the previous image. In the low exposure images, the room is dark and unclear, but the details of the windows are visible. In the high exposure images, the windows are bright and unclear, but the details of the room are revealed.

The images on the right show the interior of a church, a scene which has a variation in radiance much larger than that which can be displayed on a monitor or recorded by a conventional camera. The six individual exposures from the camera show the radiance of the scene in some range transformed to the range of brightnesses that can be displayed on a monitor. The range of radiances recorded in each photo is limited, so not all details can be displayed at once: for example, details of the dark church interior cannot be displayed at the same time as those of the bright stained-glass window. An algorithm is applied to the six images to recreate the high dynamic range radiance map of the original scene (a high dynamic range image). Alternatively, some higher-end consumer and specialist scientific digital cameras are able to record a high dynamic range image directly, for example with RAW images.

In the ideal case, a camera might measure luminance directly and store this in the HDR image; however, most high dynamic range images produced by cameras today are not calibrated or even proportional to luminance, due to practical reasons such as cost and time required to measure accurate luminance values — it is often sufficient for artists to use multiple exposures to gain an "HDR image" which grossly approximates the true luminance signal.

The high dynamic range image is passed to a tone mapping operator, in this case a non-local operator, which transforms the image into a low dynamic range image suitable for viewing on a monitor. Relative to the church interior, the stained-glass window is displayed at a much lower brightness than a linear mapping between scene radiance and pixel intensity would produce. However, this inaccuracy is perceptually less important than the image detail, which can now be shown in both the window and the church interior simultaneously.

Visual effect

Local tone mapping produces a number of characteristic effects in images. These include halos around dark objects, a "painting-like" or "cartoon-like" appearance due to a lack of large global contrasts, and highly saturated colours. Many people find the resulting images attractive and these effects to add an interesting new set of choices for post-processing in digital photography. Some people believe that the results stray too far from realism, or find them unattractive, but these are aesthetic judgements, and often concern the choices made by the photographer during the tone mapping process, rather than being a necessary consequence of using tone mapping.

Not all tone mapped images are visually distinctive. Reducing dynamic range with tone mapping is often useful in bright sunlit scenes, where the difference in intensity between direct illumination and shadow is great. In these cases the global contrast of the scene is reduced, but the local contrast maintained, while the image as a whole continues to look natural. Use of tone mapping in this context may not be apparent from the final image:

• 

  Regions of direct illumination and shadow on the Grand Canyon

• 

Tone mapping can also produce distinctive visual effects in the final image, such as the visible halo around the tower in the Cornell Law School image below. It can be used to produce these effects even when the dynamic range of the original image is not particularly high. Halos in images come about because the local tone mapping operator will brighten areas around dark objects, in order to maintain the local contrast in the original image, which fools the human visual system into perceiving the dark objects as being dark, even if their actual luminance is the same as that of areas of the image perceived as being bright. Usually this effect is subtle, but if the contrasts in the original image are extreme, or the photographer deliberately sets the luminance gradient to be very steep, the halos become visible.

• 

  Tone mapped composite image of the Cornell Law School tower in Ithaca, New York. Note the 'ringing' or 'halos' around the tower.

• 

  Nave in Old Saint Paul's, Wellington, New Zealand, using eight exposures bracketed one stop apart.

A local tone mapping algorithm without halos

(ref:A Tone Mapping Algorithm for High Contrast Images)

Tone mapping algorithms map the contrast and details of a real world scene to the limited display range of a camera. They exploit several properties of the human visual system. For Example, our eye does not perceive the actual intensity levels but can easily understand the local intensity variations (contrast) in different parts of a scene. Hence, the global intensity variations (which might correspond to large variations in intensity levels) can be suppressed without much affecting the appearance. (Ref: http://people.csail.mit.edu/fredo/PUBLI/Siggraph2002/DurandBilateral.pdf)

As per Michael Ashikhmin’s paper (Ref:A Tone Mapping Algorithm for High Contrast Images), tone mapping can be done by applying a linear mapping locally to the different regions of an image. Instead of applying a single linear mapping to the whole image (which works for limited dynamic range), local mapping is applied to small regions in which the contrast does not vary significantly. The contrast-dependent sizes of the regions (rather than using regions of fixed sizes) prevent the formation of halos around any dark edges.

The main steps of this algorithm are:

• Separate out the luminance channel.
• Find the local adaptation level by using region growing up to when the local contrast reaches a particular level.
• Apply a tone mapping function to the adaptation level to get the final pixel luminance. This compresses the local adaptation levels into the required display range.
• Do the above steps for each of the RGB and recombine these channels.
• Apply gamma correction to obtain pixels for display.

1) Estimation of the local adaptation level and local contrast

Rather than applying a single linear function for mapping the entire scene (which might work well for low dynamic range images), the algorithm suggests the application of linear mapping locally (which takes into account the problem with high dynamic range). The image is broken up into several small regions in which the contrast does not vary significantly. Local contrast can be found by computing the difference between average intensity values over a small neighborhood of each pixel. Gaussian filters are used to compute the local averages since it neglects high influences from far away regions. The size of second Gaussian window is kept twice that of the first, and the difference between the averages by the two Gaussian windows, is calculated as:

lc(s, x, y) = (Gs(L)(x, y)−G2s(L)(x, y))/Gs(L)(x, y) ........(1)

where Gs(L) represent the result of applying Gaussian filter of width s to the input image having dimensions x and y. Choosing Gs instead of G2s in the denominator for normalizing is somewhat arbitrary. lc values give the local contrast in the neighborhood of each pixel.

At first, the size of the window is kept as one pixel and the local contrast is found out. The window size is incremented iteratively up to when the local contrast reaches some particular value (a contrast of 0.5 is sufficient). As a result of this, the low contrast regions will result in larger neighborhoods (which allow for preserving of more details) and high contrast regions result in smaller neighborhoods. A limit on the window size (usually s=10) is recommended since it will prevent the system from being overwhelmed due to a very large window size. The variable window size prevents the formation of halos around dark edges.

The Gaussian average of the first window (Gs) itself can be taken as the local adaptation level La(x,y) in the neighborhood of a pixel. Then, a tone mapping function TM(L) is applied to La(x,y) to create a tone mapped adaptation image TM(La(x,y). This compresses the high dynamic range of the scene to the displayable range of the camera.

2) Tone mapping function

A linear mapping is applied to get each pixel in the displayable range of the camera. The world luminances are mapped into the display ones according to their relative positions in the corresponding perceptual scales. From this, it follows that:

(Cd(Ld) – Cd(Lmind)) / (Cd(Lmaxd) – Cd(Lmind) = (Cd(Ld) – Cd(Lmind)) / (Cd(Lmaxd) – Cd(Lmind) ........(2)

Where: C = Perceptual capacity of the human eye (described later) Subscript d is used for referring to the display L = Luminance

The perceptual capacity of the human eye is given by

C = ∆L/TVI(La) ........(3)

Where: ∆L is the just noticeable luminance by the eye, regardless of the actual luminance levels. TVI(La) is the threshold required by the eye to just notice a difference in luminance.

However, the TVI functions for cone cells and rod cells in our eye are different curves (with a common intersection point) and so an approximation is used instead for the perceptual capacity function:

  C(L)  =  L/0.0014  if  L < 0.0034                                                                  ........(4)
        =  2.4483+log(L/0.0034)/0.4027  if  0.0034 ≤ L < 1
        = 16.5630+(L−1)/0.4027  if  1 ≤ L < 7.2444
        = 32.0693+log(L/7.2444)/0.0556  otherwise

After finding C(L), the tone mapping is applied by setting Lmind = 0 and Lmaxd = LDMAX in eqn. (2). This is on the assumption that we want to use the full display range of the camera from Lmind to Lmaxd. From this, eqn. (2) becomes:

Ld = TM(L) = LDMAX (C(L) − C(Lmin)) / (C(Lmax ) − C(Lmin)) ........(5)

As the paper suggests, a typical LDMAX value that can be assumed for a camera is 100cd/m².

3) Local Contrast preservation

After performing the tone mapping operation by the above steps, the local contrast preservation is done to re-introduce details back into the scene.

Ld (x, y) = L(x, y)TM(La(x, y)) / La(x, y) ........(6)

(Ld refers to the tone mapped luminances in the display.)

4) Gamma correction and recombination of RGB channels

The RGB channels are then recombined by multiplying each of them with Ld(x,y)/L(x,y). Gamma correction with standard values of gamma is then applied to get the final displayed image.

Gallery

• 

  A tone mapped sunset. Taken as an exposure bracket of three images 2 stops apart.

• 

  HDR Tone Mapping Example

• 

  5 exposure tone mapping of the Isola Tiberina in Rome.

• 

  3 exposure (-2,0,+2) tone mapped image of a scene at Nippori Station.

• 

  Geisel Library in HDR.

• 

  HDR view from Tower Bridge in London tone-mapped from 5 exposures.

• 

  HDR view from St Paul's Cathedral in London tone-mapped from 9 exposures.

See also

• Color translation
• Gamma correction
• Tone reproduction

References

1. ^ Kate Devlin, Alan Chalmers, Alexander Wilkie, Werner Purgathofer. "STAR Report on Tone Reproduction and Physically Based Spectral Rendering" in Eurographics 2002. DOI: 10.1145/1073204.1073242
2. ^ Raanan Fattal, Dani Lischinski, Michael Werman. "Gradient Domain High Dynamic Range Compression"
3. ^ Rafal Mantiuk, Karol Myszkowski, Hans-Peter Seidel. "A Perceptual Framework for Contrast Processing of High Dynamic Range Images"
4. ^ Alan Gilchrist. "An Anchoring Theory of Lightness Perception".
5. ^ Grzegorz Krawczyk, Karol Myszkowski, Hans-Peter Seidel. "Lightness Perception in Tone Reproduction for High Dynamic Range Images"

External links

• CVLTonemap: GPU accelerated tone mapping
• HDR Darkroom
• pfstmo: implementation of tone mapping operators
• exrtools: a collection of utilities for manipulating OpenEXR images (includes some tone mapping operators)
• pfstools is an open-source set of command line programs for reading, writing and manipulating high-dynamic range (HDR) images and video frames
• Luminance HDR/QtPfsGui is a free (open-source) HDR-workflow software for Linux, Windows and Mac OS X based around the pfstools package
• LDR tonemapping is a free (open-source) tonemapper for low dynamic range images (a.k.a. "pseudo-HDR")
• Atlas is a free (open-source) port of the pfstmo tone mapping operators to Adobe After Effects
• Flickr HDR pool, a collection of surreal tone mappings
• UC Berkeley paper with raw data for Purkinje effect
• Stuck in Customs, an extensive tutorial to make HDR images

Tone mapping algorithms

• Perceptually Based Tone Mapping for Low-Light Conditions
• Photographic Tone Reproduction for Digital Images
• Lightness Perception in Tone Reproduction for High Dynamic Range Images
• Contrast Processing of High Dynamic Range Images
• Fast Bilateral Filtering for the Display of High-Dynamic-Range Images
• A Fast Approximation of the Bilateral Filter using a Signal Processing Approach
• Gradient Domain High Dynamic Range Compression
• A tone mapping algorithm for high contrast images