Multi-exposure HDR capture
This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these template messages)
In photography and videography, multi-exposure HDR capture is a technique allowing to capture high dynamic range (HDR) images by taking and then combining several different exposures of the same subject matter.
Images captured by cameras allow differentiation only within a certain range of luminosity. Outside this range, no features are visible because everything appears pure white in the brighter areas and pure black in the darker areas. The ratio between the maximum and the minimum of the tonal value in an image is known as the dynamic range. Combining several different, narrower range, exposures results in an image with a greater dynamic range than what is possible by taking one single exposure. HDR is useful for recording many real-world scenes containing very bright, direct sunlight to extreme shade, or very faint nebulae.
The term "HDR" may also refer to the overall process of capturing HDR images from multiple exposures. Many smartphones have a built-in HDR feature performing the process in a automated way for photo capture. This HDR technique can also be used to capture video by taking and combining multiple exposures for each frame of the video.
Due to the limitations of printing and display contrast, the extended luminosity range of input HDR images has to be compressed to be made visible. The method of rendering an HDR image to a standard monitor or printing device is called tone mapping. This method reduces the overall contrast of an HDR image to facilitate display on devices or printouts with lower dynamic range. It can be applied to produce images with preserved local contrast or exaggerated local contrast for artistic effect.
One aim of HDR is to present a similar range of luminance to that experienced through the human visual system. The human eye, through non-linear response, adaptation of the iris, and other methods, adjusts constantly to a broad range of luminance present in the environment. The brain continuously interprets this information so that a viewer can see in a wide range of light conditions.
|Human eye: close objects||7.5||150...200|
|Human eye: 4° angular separation||13||8000...10000|
|Human eye (static)||10...14 ||1000...15000|
|Negative film (Kodak VISION3)||13 ||8000|
|1/1.7" camera (Nikon Coolpix P340)||11.9 ||3800|
|1" camera (Canon PowerShot G7 X)||12.7 ||6600|
|Four-thirds DSLR camera (Panasonic Lumix DC-GH5)||13.0 ||8200|
|APS DSLR camera (Nikon D7200)||14.6 ||24800|
|Full-frame DSLR camera (Nikon D810)||14.8 ||28500|
Most cameras cannot provide this range of exposure values within a single exposure, due to their low dynamic range. Standard photographic and image techniques allow differentiation only within a certain range of brightness. Outside of this range, no features are visible because there is no differentiation in bright areas as everything appears just pure white, and there is no differentiation in darker areas as everything appears pure black. Non-HDR cameras take photographs with a limited exposure range, referred to as low dynamic range (LDR), resulting in the loss of detail in highlights or shadows.
HDR imaging is used in extreme dynamic range applications like welding or automotive work. In security cameras the term used instead of HDR is "wide dynamic range".
In photography, dynamic range is measured in exposure value (EV) differences, known as stops.
High-dynamic-range photographs are generally achieved by capturing multiple standard-exposure images, often using exposure bracketing, and then later merging them into a single HDR image, usually within a photo manipulation program.
Capturing multiple images (exposure bracketing)
Any camera that allows manual exposure control can make images for HDR work, although one equipped with auto exposure bracketing (AEB) is far better suited. Images from film cameras are less suitable as they often must first be digitized, so that they can later be processed using software HDR methods.
An increase of one EV, or one stop, represents a doubling of the amount of light. Conversely, a decrease of one EV represents a halving of the amount of light. Therefore, revealing detail in the darkest of shadows requires high exposures, while preserving detail in very bright situations requires very low exposures.
In most imaging devices, the degree of exposure to light applied to the active element (be it film or CCD) can be altered in one of two ways: by either increasing/decreasing the size of the aperture or by increasing/decreasing the time of each exposure. Exposure variation in an HDR set is only done by altering the exposure time and not the aperture size; this is because altering the aperture size also affects the depth of field and so the resultant multiple images would be quite different, preventing their final combination into a single HDR image.
An important limitation for HDR photography is that any movement between successive images will impede or prevent success in combining them afterward. Also, as one must create several images (often three or five and sometimes more) to obtain the desired luminance range, such a full set of images takes extra time. HDR photographers have developed calculation methods and techniques to partially overcome these problems, but the use of a sturdy tripod is, at least, advised.
Color film negatives and slides consist of multiple film layers that respond to light differently. Original film (especially negatives versus transparencies or slides) feature a very high dynamic range (in the order of 8 for negatives and 4 to 4.5 for slides).
Merging the images into an HDR image
This section is empty. You can help by adding to it. (November 2021)
Information stored in high-dynamic-range images typically corresponds to the physical values of luminance or radiance that can be observed in the real world. This is different from traditional digital images, which represent colors as they should appear on a monitor or a paper print. Therefore, HDR image formats are often called scene-referred, in contrast to traditional digital images, which are device-referred or output-referred. Furthermore, traditional images are usually encoded for the human visual system (maximizing the visual information stored in the fixed number of bits), which is usually called gamma encoding or gamma correction. The values stored for HDR images are often gamma compressed (power law) or logarithmically encoded, or floating-point linear values, since fixed-point linear encodings are increasingly inefficient over higher dynamic ranges.
HDR images often don't use fixed ranges per color channel—other than traditional images—to represent many more colors over a much wider dynamic range (multiple channels). For that purpose, they do not use integer values to represent the single color channels (e.g., 0–255 in an 8 bit per pixel interval for red, green and blue) but instead use a floating point representation. Common are 16-bit (half precision) or 32-bit floating-point numbers to represent HDR pixels. However, when the appropriate transfer function is used, HDR pixels for some applications can be represented with a color depth that has as few as 10–12 bits for luminance and 8 bits for chrominance without introducing any visible quantization artifacts.
Tone mapping reduces the dynamic range, or contrast ratio, of an entire image while retaining localized contrast. Although it is a distinct operation, tone mapping is often applied to HDRI files by the same software package.
Tone mapping is often need because the dynamic range of the electronic representation that display can receive is often lower than the dynamic range of the captured image. HDR displays can receive a higher dynamic range signal than SDR displays, reducing the need for tone mapping.
Several software applications are available on the PC, Mac and Linux platforms for producing HDR files and tone mapped images. Notable titles include:
As the popularity of this imaging method grows, several camera manufacturers are now offering built-in HDR features. For example, the Pentax K-7 DSLR has an HDR mode that captures an HDR image and outputs (only) a tone mapped JPEG file. The Canon PowerShot G12, Canon PowerShot S95, and Canon PowerShot S100 offer similar features in a smaller format. Nikon's approach is called 'Active D-Lighting' which applies exposure compensation and tone mapping to the image as it comes from the sensor, with the emphasis being on creating a realistic effect.
Some of the sensors on modern phones and cameras may even combine the two images on-chip so that a wider dynamic range without in-pixel compression is directly available to the user for display or processing.
This is an example of four standard dynamic range images that are combined to produce three resulting tone mapped images:
- Exposured images:
- Results after processing:
This is an example of a scene with a very wide dynamic range:
- Exposured images:
- Results after processing:
A fast-moving subject (or unsteady camera) will result in a "ghost" effect or a staggered-blur strobe effect, as a result of the merged images not being identical, but each capturing the moving subject at a different moment in time, with its position changed. Sudden changes in the lighting conditions (strobed LED light) can also interfere with the desired results, by producing one or more HDR layers that do have the luminosity expected by an automated HDR system, though one might still be able to produce a reasonable HDR image manually in software by rearranging the image layers to merge in order of their actual luminosity.
Because of the nonlinearity of some sensors image artifacts can be common.
Although not as established as for still photography capture, it is also possible to capture and combine multiple images for each frame of a video in order to increase the dynamic range captured by the camera. This can be done via multiple methods:
- A time-lapse of individually images created via the HDR technique.
- Taking consecutively two differently exposed images by cutting the frame rate in half.
- Taking simultaneously two differently exposed images by cutting the resolution in half.
- Taking simultaneously two differently exposed images with full resolution and frame rate via a sensor with dual gain architecture. For example: Arri Alexa's sensor, Samsung sensors with Smart-ISO Pro.
Some cameras designed for use in security applications can automatically provide two or more images for each frame, with changing exposure. For example, a sensor for 30fps video will give out 60fps with the odd frames at a short exposure time and the even frames at a longer exposure time.
Mid 19th century
The idea of using several exposures to adequately reproduce a too-extreme range of luminance was pioneered as early as the 1850s by Gustave Le Gray to render seascapes showing both the sky and the sea. Such rendering was impossible at the time using standard methods, as the luminosity range was too extreme. Le Gray used one negative for the sky, and another one with a longer exposure for the sea, and combined the two into one picture in positive.
Mid 20th century
|Schweitzer at the Lamp, by W. Eugene Smith|
Manual tone mapping was accomplished by dodging and burning – selectively increasing or decreasing the exposure of regions of the photograph to yield better tonality reproduction. This was effective because the dynamic range of the negative is significantly higher than would be available on the finished positive paper print when that is exposed via the negative in a uniform manner. An excellent example is the photograph Schweitzer at the Lamp by W. Eugene Smith, from his 1954 photo essay A Man of Mercy on Albert Schweitzer and his humanitarian work in French Equatorial Africa. The image took five days to reproduce the tonal range of the scene, which ranges from a bright lamp (relative to the scene) to a dark shadow.
Ansel Adams elevated dodging and burning to an art form. Many of his famous prints were manipulated in the darkroom with these two methods. Adams wrote a comprehensive book on producing prints called The Print, which prominently features dodging and burning, in the context of his Zone System.
With the advent of color photography, tone mapping in the darkroom was no longer possible due to the specific timing needed during the developing process of color film. Photographers looked to film manufacturers to design new film stocks with improved response, or continued to shoot in black and white to use tone mapping methods.
Color film capable of directly recording high-dynamic-range images was developed by Charles Wyckoff and EG&G "in the course of a contract with the Department of the Air Force". This XR film had three emulsion layers, an upper layer having an ASA speed rating of 400, a middle layer with an intermediate rating, and a lower layer with an ASA rating of 0.004. The film was processed in a manner similar to color films, and each layer produced a different color. The dynamic range of this extended range film has been estimated as 1:108. It has been used to photograph nuclear explosions, for astronomical photography, for spectrographic research, and for medical imaging. Wyckoff's detailed pictures of nuclear explosions appeared on the cover of Life magazine in the mid-1950s.
Late 20th century
Georges Cornuéjols and licensees of his patents (Brdi, Hymatom) introduced the principle of HDR video image, in 1986, by interposing a matricial LCD screen in front of the camera's image sensor, increasing the sensors dynamic by five stops.
The concept of neighborhood tone mapping was applied to video cameras in 1988 by a group from the Technion in Israel, led by Oliver Hilsenrath and Yehoshua Y. Zeevi. Technion researchers filed for a patent on this concept in 1991, and several related patents in 1992 and 1993.
In February and April 1990, Georges Cornuéjols introduced the first real-time HDR camera that combined two images captured successively by a sensor or simultaneously by two sensors of the camera. This process is known as bracketing used for a video stream.
In 1991, the first commercial video camera was introduced that performed real-time capturing of multiple images with different exposures, and producing an HDR video image, by Hymatom, licensee of Georges Cornuéjols.
Also in 1991, Georges Cornuéjols introduced the HDR+ image principle by non-linear accumulation of images to increase the sensitivity of the camera: for low-light environments, several successive images are accumulated, thus increasing the signal-to-noise ratio.
In 1993, another commercial medical camera producing an HDR video image, by the Technion.
Modern HDR imaging uses a completely different approach, based on making a high-dynamic-range luminance or light map using only global image operations (across the entire image), and then tone mapping the result. Global HDR was first introduced in 1993 resulting in a mathematical theory of differently exposed pictures of the same subject matter that was published in 1995 by Steve Mann and Rosalind Picard.
On October 28, 1998, Ben Sarao created one of the first nighttime HDR+G (high dynamic range + graphic) image of STS-95 on the launch pad at NASA's Kennedy Space Center. It consisted of four film images of the space shuttle at night that were digitally composited with additional digital graphic elements. The image was first exhibited at NASA Headquarters Great Hall, Washington DC, in 1999 and then published in Hasselblad Forum.
The advent of consumer digital cameras produced a new demand for HDR imaging to improve the light response of digital camera sensors, which had a much smaller dynamic range than film. Steve Mann developed and patented the global-HDR method for producing digital images having extended dynamic range at the MIT Media Lab. Mann's method involved a two-step procedure: First, generate one floating point image array by global-only image operations (operations that affect all pixels identically, without regard to their local neighborhoods). Second, convert this image array, using local neighborhood processing (tone-remapping, etc.), into an HDR image. The image array generated by the first step of Mann's process is called a lightspace image, lightspace picture, or radiance map. Another benefit of global-HDR imaging is that it provides access to the intermediate light or radiance map, which has been used for computer vision, and other image processing operations.
In February 2001, the Dynamic Ranger technique was demonstrated, using multiple photos with different exposure levels to accomplish high dynamic range similar to the naked eye.
In the early 2000s, several scholarly research efforts used consumer-grade sensors and cameras. A few companies such as RED and Arri have been developing digital sensors capable of a higher dynamic range. RED EPIC-X can capture time-sequential HDRx images with a user-selectable 1–3 stops of additional highlight latitude in the "x" channel. The "x" channel can be merged with the normal channel in post production software. The Arri Alexa camera uses a dual-gain architecture to generate an HDR image from two exposures captured at the same time.
With the advent of low-cost consumer digital cameras, many amateurs began posting tone-mapped HDR time-lapse videos on the Internet, essentially a sequence of still photographs in quick succession. In 2010, the independent studio Soviet Montage produced an example of HDR video from disparately exposed video streams using a beam splitter and consumer grade HD video cameras. Similar methods have been described in the academic literature in 2001 and 2007.
- Comparison of graphics file formats
- HDRi (data format)
- High-dynamic-range rendering
- High-dynamic-range video
- JPEG XT
- Logluv TIFF
- RGBE image format
- scRGB colorspace
- Ultra-high-definition television
- Wide dynamic range
- "Compositing Multiple Pictures of the Same Scene", by Steve Mann, in IS&T's 46th Annual Conference, Cambridge, Massachusetts, May 9–14, 1993
- Mann, S.; Picard, R. W. "On Being 'Undigital' with Digital Cameras: Extending Dynamic Range by Combining Differently Exposed Pictures" (PDF).
- Reinhard, Erik; Ward, Greg; Pattanaik, Sumanta; Debevec, Paul (2005). High Dynamic Range Imaging: Acquisition, Display, and Image-based Lighting. Amsterdam: Elsevier / Morgan Kaufmann. p. 7. ISBN 978-0-12-585263-0.
Images that store a depiction of the scene in a range of intensities commensurate with the scene are what we call HDR, or 'radiance maps'. On the other hand, we call images suitable for display with current display technology LDR.
- Banterle, Francesco; Artusi, Alessandro; Debattista, Kurt; Chalmers, Alan (2011). Advanced High Dynamic Range Imaging: Theory and Practice. AK Peters / CRC Press. ISBN 978-156881-719-4.
- McHugh, Sean, ed. (2005). "Dynamic Range in Digital Photography". Cambridge in Colour. Retrieved December 30, 2010.
- "Dynamic Range".[permanent dead link]
- "Camera Sensor Ratings". DxOMark. DxO Labs. 2015. Retrieved February 2, 2015.
- Darmont, Arnaud (2012). High Dynamic Range Imaging: Sensors and Architectures (First ed.). SPIE press. ISBN 978-0-81948-830-5.
- "Auto Exposure Bracketing Settings by Camera Model". HDR Photography Resources. February 28, 2016. Retrieved June 12, 2020.
- Ward, Greg. "High Dynamic Range Image Encodings". Anyhere.com. Anyhere Software.
- "The Radiance Picture File Format". RadSite.LBL.gov. Lawrence Berkeley National Laboratory. Archived from the original on January 28, 2019. Retrieved June 12, 2020.
- Fernando, Randima (2004). "26.5 Linear Pixel Values". GPU Gems. Boston: Addison-Wesley. ISBN 0-321-22832-4. Archived from the original on April 12, 2010 – via Developer.Nvidia.com.
- Mantiuk, Rafal; Krawczyk, Grzegorz; Myszkowski, Karol; Seidel, Hans-Peter. "Perception-motivated High Dynamic Range Video Encoding". Resources.MPI-Inf.MPG.de. Max Planck Institute for Informatics.
- Howard, Jack (May 20, 2009). "The Pentax K-7: The Era of In-camera High Dynamic Range Imaging Has Arrived!". Adorama Learning Center. Adorama. Archived from the original on December 23, 2014. Retrieved 18 August 2009.
- Mokey, Nick (September 14, 2010). "Canon PowerShot G12 picks up HD video recording, built-in HDR". Digital Trends. Retrieved June 12, 2020.
- Heiner, Steve (2017). "Intermediate: Balancing Photo Exposures with Active D-lighting". "Ideas and Inspiration" section. Nikon Learn and Explore. Nikon. Retrieved August 2, 2017.
- Android examples: "Apps: HDR mode". Google Play. Retrieved June 12, 2020.
- Sá, Asla M.; Carvalho, Paulo Cezar; Velho, Luiz (2007). High Dynamic Range. Focal Press. p. 11. ISBN 978-1-59829-562-7.
- "RED.com". www.red.com. Retrieved 2021-11-05.
- "Create HDR time-lapse video with a digital camera". Macworld. 2011-02-07. Retrieved 2021-11-06.
- "Sony IMX378: Comprehensive Breakdown of the Google Pixel's Sensor and its Features". xda-developers. 2016-10-12. Retrieved 2021-11-05.
- "ARRI Group: ALEXA's Sensor". www.arri.com. Archived from the original on 1 August 2016. Retrieved 2 July 2016.
- "[Video] Painting With Light: How Smart-ISO Pro Captures Lifelike HDR Images". news.samsung.com. Retrieved 2021-11-05.
- "Qualcomm explains how the Snapdragon 888 is changing the camera game (Video!)". Android Authority. 2020-12-04. Retrieved 2021-06-08.
- Rehm, Lars (2021-04-02). "Xiaomi Mi 11 Ultra Camera review: Large sensor power". DXOMARK. Retrieved 2021-06-08.
- "Gustave Le Gray, Photographer". Getty.edu. J. Paul Getty Museum. July 9 – September 29, 2002. Retrieved September 14, 2008.
- The Future of Digital Imaging – High Dynamic Range Photography, Jon Meyer, Feb 2004
- 4.209: The Art and Science of Depiction, Frédo Durand and Julie Dorsey, Limitations of the Medium: Compensation and accentuation – The Contrast is Limited, lecture of Monday, April 9. 2001, slide 57–59; image on slide 57, depiction of dodging and burning on slide 58
- US -number=3450536, Wyckoff, Charles W. & EG&G Inc., assignee, "Silver Halide Photographic Film having Increased Exposure-response Characteristics", published 1969-09-17
- Wyckoff, Charles W. (June–July 1962). "Experimental extended exposure response film". Society of Photographic Instrumentation Engineers Newsletter: 16–20.
- Goesele, Michael; et al. "High Dynamic Range Techniques in Graphics: from Acquisition to Display" (PDF). Eurographics 2005 Tutorial T7. Max Planck Institute for Informatics.
- "The Militarily Critical Technologies List" (PDF). FAS.org. Intelligence Resource Program, Federation of American Scientists. 1998. pp. II-5-100, II-5-107. Retrieved June 12, 2020.
- Young, Andrew T.; Boeschenstein, Harold, Jr. (1964). Isotherms in the Region of Proclus at a Phase Angle of 9.8 Degrees. Scientific Report series. No. 5. Cambridge, Massachusetts: College Observatory, Harvard University.
- Bryant, R. L.; Troup, G. J.; Turner, R. G. (1965). "The use of a high-intensity-range photographic film for recording extended diffraction patterns and for spectrographic work". Journal of Scientific Instruments. 42 (2): 116. Bibcode:1965JScI...42..116B. doi:10.1088/0950-7671/42/2/315.
- Eber, Leslie M.; Greenberg, Haervey M.; Cooke, John M.; Gorlin, Richard (1969). "Dynamic Changes in Left Ventricular Free Wall Thickness in the Human Heart". Circulation. 39 (4): 455–464. doi:10.1161/01.CIR.39.4.455. PMID 5778246.
- "Image processing device for controlling the transfer function of an optical system". Worldwide.Espacenet.com. Espacenet.
- US patent 5144442, Ginosar, Ran; Hilsenrath, Oliver & Zeevi, Yehoshua Y., "Wide dynamic range camera", published 1992-09-01
- Ginosar, Ran; Zinaty, Ofra; Sorek, Noam; Genossar, Tamar; Zeevi, Yehoshua Y.; Kligler, Daniel J.; Hilsenrath, Oliver (1993). "Adaptive Sensitivity". VISL.Technion.ac.il. Vision and Image Sciences Laboratory, Technion, Israel Institute of Technology. Archived from the original on September 7, 2014. Retrieved January 27, 2019.
- "Device for increasing the dynamic range of a camera". Worldwide.Espacenet.com. Espacenet.
- "Camera with very wide dynamic range". Worldwide.Espacenet.com. Espacenet.
- Sarao, Ben M. (1999). Gunnarsson, S. (ed.). Ben Sarao, Trenton, NJ. Hasselblad Forum. 35. ISSN 0282-5449.
- US application 5828793, Mann, Steve, "Method and apparatus for producing digital images having extended dynamic ranges", published 1998-10-27
- "Dynamic Ranger".
- Kang, Sing Bing; Uyttendaele, Matthew; Winder, Simon; Szeliski, Richard (2003). ACM SIGGRAPH 2003 Papers – on SIGGRAPH '03. ch. High dynamic range video (pages 319–325). doi:10.1145/1201775.882270. ISBN 978-1-58113-709-5. S2CID 13946222.
- "RED Digital Cinema | 8K & 5K Professional Cameras". Archived from the original on 27 July 2016. Retrieved 27 July 2016.
- "ARRI | Inspiring your Vision". Archived from the original on 8 September 2011. Retrieved 23 January 2021.
- "Sony IMX378: Comprehensive Breakdown of the Google Pixel's Sensor and its Features". xda-developers. 2016-10-12. Archived from the original on 2019-04-01. Retrieved 2016-10-17.
- "HDR video accomplished using dual 5D Mark IIs, is exactly what it sounds like". Engadget. Archived from the original on 14 June 2017. Retrieved 29 August 2017.
- "A Real Time High Dynamic Range Light Probe". Archived from the original on 17 June 2016. Retrieved 27 July 2016.
- McGuire, Morgan; Matusik, Wojciech; Pfister, Hanspeter; Chen, Billy; Hughes, John; Nayar, Shree (2007). "Optical Splitting Trees for High-Precision Monocular Imaging". IEEE Computer Graphics and Applications. 27 (2): 32–42. doi:10.1109/MCG.2007.45. PMID 17388201. S2CID 3055332. Archived from the original on 23 January 2021. Retrieved 14 July 2019.
- Reichmann, Michael (2005). "Merge to HDR in Photoshop CS2: A First Look". The Luminous Landscape. Archived from the original on January 2, 2010. Retrieved August 27, 2009.
- Anderson, Kareem (June 30, 2016). "Microsoft talks up the advantages of HDR photography and videography in Universal Windows Platform apps". OnMSFT.com. Retrieved June 12, 2020.
- Media related to High-dynamic-range imaging at Wikimedia Commons