Jump to content

Light field camera

From Wikipedia, the free encyclopedia

This is an old revision of this page, as edited by Monkbot (talk | contribs) at 12:50, 2 January 2021 (Task 18 (cosmetic): eval 11 templates: hyphenate params (2×);). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

Lytro Illum 2nd generation light field camera.
Front and back of a Lytro, the first consumer light field camera, showing the front lens and LCD touchscreen.

A light field camera, also known as plenoptic camera, captures information about the light field emanating from a scene; that is, the intensity of light in a scene, and also the direction that the light rays are traveling in space. This contrasts with a conventional camera, which records only light intensity.

One type of light field camera uses an array of micro-lenses placed in front of an otherwise conventional image sensor to sense intensity, color, and directional information. Multi-camera arrays are another type of light field camera. Holograms are a type of film-based light field image.

Technology

Early research

The first light field camera was proposed by Gabriel Lippmann in 1908. He called his concept "integral photography". Lippmann's experimental results included crude integral photographs made by using a plastic sheet embossed with a regular array of microlenses, or by partially embedding very small glass beads, closely packed in a random pattern, into the surface of the photographic emulsion.

In 1992, Adelson and Wang proposed the design of a plenoptic camera that can be used to significantly reduce the correspondence problem in stereo matching.[1] To achieve this, an array of microlenses is placed at the focal plane of the camera main lens. The image sensor is positioned slightly behind the microlenses. Using such images, the displacement of image parts that are not in focus can be analyzed and depth information can be extracted.

Standard plenoptic camera

This demonstrates the capability of changing the focal distance and depth of field after a photo is taken - Near focus (top), Far focus (middle), Full depth of field (bottom) - using the Lytro Illum light field camera software.

The "standard plenoptic camera" is a standardized mathematical model used by researchers to compare different types of plenoptic (or light field) cameras. By definition the "standard plenoptic camera" has microlenses placed one focal length away from the image plane of a sensor.[2][3][4] Research has shown that its maximum baseline is confined to the main lens entrance pupil size which proves to be small compared to stereoscopic setups.[1][5] This implies that the "standard plenoptic camera" may be intended for close range applications as it exhibits increased depth resolution at very close distances that can be metrically predicted based on the camera's parameters.[6]

In 2004, a team at Stanford University Computer Graphics Laboratory used a 16-megapixel camera with a 90,000-microlens array (meaning that each microlens covers about 175 pixels, and the final resolution is 90 kilopixels) to demonstrate that pictures can be refocused after they are taken.[2]

Focused plenoptic camera

Lumsdaine and Georgiev described the design of a type of plenoptic camera in which the microlens array can be positioned before or behind the focal plane of the main lens. This modification samples the light field in a way that trades angular resolution for higher spatial resolution. With this design, images can be post focused with a much higher spatial resolution than with images from the standard plenoptic camera. However, the lower angular resolution can introduce some unwanted aliasing artifacts.

Coded aperture camera

A type of plenoptic camera using a low-cost printed film mask instead of a microlens array was proposed by researchers at MERL in 2007.[7] This design overcomes several limitations of microlens arrays in terms of chromatic aberrations and loss of boundary pixels, and allows higher-spatial-resolution photos to be captured. However the mask-based design reduces the amount of light that reaches the image sensor compared to cameras based on microlens arrays.

Stereo with plenoptic cameras

Plenoptic cameras are good for imaging fast moving objects where auto focus may not work well, and for imaging objects where auto focus is not affordable or usable such as with security cameras.[8] A recording from a security camera based upon plenoptic technology could be used to produce an accurate 3D model of a subject.[9]

Manufacturers

Cameras available for purchase

Lytro was founded by Stanford University Computer Graphics Laboratory alumnus Ren Ng to commercialize the light field camera he developed as a graduate student there. Lytro has developed consumer light field digital cameras capable of capturing images using a plenoptic technique.[10] After Lytro ceased operations in March 2018, few options exist for purchasing light-field cameras.

Raytrix has sold several models of plenoptic cameras for industrial and scientific applications since 2010, with field of view starting from 1 megapixel.[11][12]

d'Optron and Rebellion Photonics sell several plenoptic cameras, specializing in microscopy and gas leak detection, respectively.

Other cameras

Pelican Imaging has thin multi-camera array systems intended for consumer electronics. Pelican's systems use from 4 to 16 closely spaced micro-cameras instead of a micro-lens array image sensor.[13] Nokia has invested in Pelican Imaging to produce a plenoptic camera system with 16-lens array camera expected to be implemented in Nokia smartphones in 2014.[14] More recently, Pelican has moved to designing supplementary cameras that add depth-sensing capabilities to a device's main camera, rather than stand-alone array cameras.[15]

The Adobe light field camera is a prototype 100-megapixel camera that takes a three-dimensional photo of the scene in focus using 19 uniquely configured lenses. Each lens will take a 5.2-megapixel photo of the entire scene around the camera and each image can be focused later in any way.[16]

The CAFADIS camera is a plenoptic camera developed by the University of La Laguna (Spain).[17] CAFADIS stands (in Spanish) for phase-distance camera, since it can be used for distance and optical wavefront estimation. From a single shot it can produce several images refocused at different distances, depth maps, all-in-focus images and stereo pairs. A similar optical design can also be used in adaptive optics in astrophysics, in order to correct the aberrations caused by atmospheric turbulence in telescope images. In order to perform these tasks, different algorithms, running on GPU and FPGA, operate on the raw image captured by the camera.

Mitsubishi Electric Research Laboratories's (MERL) light field camera[7] is based on the principle of optical heterodyning and uses a printed film (mask) placed close to the sensor. Any hand-held camera can be converted into a light field camera using this technology by simply inserting a low-cost film on top of the sensor.[18] A mask-based design avoids the problem of loss of resolution, since a high-resolution photo can be generated for the focused parts of the scene.

Amateur light field cameras

The modification of standard digital cameras requires little more than the capacity to produce suitable sheets of micro-lens material, hence a number of hobbyists have been able to produce cameras whose images can be processed to give either selective depth of field or direction information.[19]

Use in education

In a study published in 2017, researchers observed that incorporation of light field photographed images into an online anatomy module did not result in better learning outcomes compared to an identical module with traditional photographs of dissected cadavers.[20]

Light field microscope

Stanford University Computer Graphics Laboratory has developed a light field microscope using a microlens array similar to the one used in the light field camera developed by the lab. The prototype is built around a Nikon Eclipse transmitted light microscope/wide-field fluorescence microscope and standard CCD cameras. Light field capturing ability is obtained by a module containing a microlens array and other optical components placed in the light path between the objective lens and camera, with the final multifocused image rendered using deconvolution.[21][22][23] A later version of the prototype added a light field illumination system consisting of a video projector (allowing computational control of illumination) and a second microlens array in the illumination light path of the microscope. The addition of a light field illumination system both allowed for additional types of illumination (such as oblique illumination and quasi-dark-field) and correction for optical aberrations.[22]

See also

References

  1. ^ a b Adelson, E. H.; Wang, J. Y. A. (1992). "Single Lens Stereo with Plenoptic Camera". IEEE Transactions on Pattern Analysis and Machine Intelligence. 14 (2): 99–106. CiteSeerX 10.1.1.53.7845. doi:10.1109/34.121783.
  2. ^ a b R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan. Light Field Photography with a Hand-Held Plenoptic Camera. Stanford University Computer Science Tech Report CSTR 2005-02, April 2005.
  3. ^ Lumsdaine, A., Georgiev, T., The Focused Plenoptic Camera, ICCP, April 2009.
  4. ^ Hahne, C.; Aggoun, A.; Velisavljevic, V.; Fiebig, S.; Pesch, M. (2016). "Refocusing distance of a standard plenoptic camera". Optics Express. 24 (19): 21521–21540. Bibcode:2016OExpr..2421521H. doi:10.1364/oe.24.021521. hdl:10547/622011. PMID 27661891.
  5. ^ Hahne, C.; Aggoun, A.; Velisavljevic, V.; Fiebig, S.; Pesch, M. (2017). "Baseline and Triangulation Geometry in a Standard Plenoptic Camera" (PDF). Int. J. Comput. Vis.
  6. ^ Light field geometry estimator
  7. ^ a b Ashok Veeraraghavan, Ramesh Raskar, Amit Agrawal, Ankit Mohan and Jack Tumblin. Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing. ACM Transactions on Graphics, Vol. 26, Issue 3, July 2007.
  8. ^ Polydioptric Camera Design, says good for following moving objects.
  9. ^ Computer scientists create a 'light field camera' that banishes fuzzy photos, Anne Strehlow. Stanford Report. November 3, 2005.
  10. ^ "Lytro website". Archived from the original on 2011-11-04. Retrieved 2011-10-30.
  11. ^ One Camera With 40,000 Lenses Helps Prevent Blurry Images. Popular Science May 2011.
  12. ^ The First Plenoptic Camera on the Market
  13. ^ http://www.pelicanimaging.com/technology/paper.html.
  14. ^ "Pelican Imaging's 16-lens array camera coming to smartphones next year". May 2, 2013.
  15. ^ Koifman, Vladimir (2015-07-25). "Pelican Imaging Layoffs?". Image Sensors World. Archived from the original on 26 November 2019. Retrieved 2015-11-17.
  16. ^ Keats, Jonathon; Holland, Kris; McLeod, Gary. "PopSci's How It Works – 100 Megapixel Camera". PopSci.com. Popular Science. Archived from the original (Adobe Flash) on 2008-01-17. Retrieved 26 July 2009.
  17. ^ "CAFADIS - University of la Laguna". Archived from the original on 26 November 2019.
  18. ^ http://www.umiacs.umd.edu/~aagrawal/sig08/BuildingLightFieldCamera.html Archived 2013-12-31 at the Wayback Machine.
  19. ^ home build plenoptic camera
  20. ^ Pascoe, Michael A.; Lee, Lisa M.J. (September 2017). "Incorporation of Light Field Photography into an Online Anatomy Resource Does Not Influence Student Quiz Performance or Perceptions of Usability". Medical Science Educator. 27 (3): 465–474. doi:10.1007/s40670-017-0410-8. ISSN 2156-8650. S2CID 148803076.
  21. ^ Levoy, M; Ng, R; Adams, A; Footer, M; Horowitz, M (2006). "Light Field Microscopy". ACM Transactions on Graphics. 25 (3): 924–93. doi:10.1145/1141911.1141976.
  22. ^ a b Levoy, M; Zhang, Z; McDowall, I (2009). "Recording and controlling the 4D light field in a microscope". Journal of Microscopy. 235 (2): 144–162. CiteSeerX 10.1.1.163.269. doi:10.1111/j.1365-2818.2009.03195.x. PMID 19659909. S2CID 13194109.
  23. ^ Levoy M. 2008. Stanford Light Field Microscope Project (web page). Stanford Computer Graphics Laboratory.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Light_field_camera&oldid=997831971"