A light-field camera, also called a plenoptic camera, is a camera that uses a microlens array to capture 4D light field information about a scene. Such light field information can be used to improve the solution of computer graphics and computer vision-related problems, and to make digital plenoptic pictures that can be refocused after they are taken.
The first light-field camera was proposed by Gabriel Lippmann in 1908, which used integral photography as the underlying technology. In 1992, Adelson and Wang proposed the design of a plenoptic camera that can be used to significantly reduce the correspondence problem in stereo matching. To achieve this, an array of microlenses is placed at the focal plane of the camera main lens. The image sensor is positioned slightly behind the microlenses. Using such images the displacement of image parts that are not in focus can be analyzed and depth information can be extracted.
Potentially, this camera system can be used to refocus an image virtually on a computer after the picture has been taken, as explained by Ng et al. The drawback of such a system is the low resolution that the final images have. As one microlens samples the light directions at one spatial point an increase in the number of image pixels can only be done by increasing the number of microlenses by the same amount.
To overcome this limitation, Lumsdaine and Georgiev describe a new design of a plenoptic camera, called focused plenoptic camera where the microlens array is positioned before or behind the focal plane of the main lens. This modification samples the light field in a different way that allows for a higher spatial resolution by having a lower angular resolution at the same time. With this design images can be refocused with a much higher spatial resolution. However, the low angular resolution can introduce some unwanted aliasing artifacts.
A different design using low-cost printed film (mask) instead of microlens array was proposed by researchers at MERL in 2007. This design overcomes several limitations of microlens array in terms of chromatic aberrations, loss of boundary pixels, and allows higher-spatial-resolution photos to be captured. However, a mask-based design reduces light compared to microlens arrays.
In 2004, a team at Stanford University Computer Graphics Laboratory used a 16-megapixel camera with a 90,000-microlens array (meaning that each microlens covers about 175 pixels, and the final resolution is 90 kilopixels) to demonstrate that pictures can be refocused after they are taken.
Stereo with plenoptic cameras
Manufacturers of light-field cameras
Cameras available for purchase
Lytro was founded by Stanford University Computer Graphics Laboratory alumnus Ren Ng to commercialize the light-field camera he developed as a graduate student there. Lytro has developed a consumer light-field "digital camera capable of images using a plenoptic technique.
The Adobe light-field camera is a prototype 100-megapixel camera that takes a three-dimensional photo of the scene in focus using 19 uniquely configured lenses. Each lens will take a 5.2-megapixel photo of the entire scene around the camera and each image can be focused later in any way.
The CAFADIS camera is a plenoptic camera developed by the University of La Laguna (Spain). CAFADIS stands (in Spanish) for phase-distance camera, since it can be used for distance and optical wavefront estimation. From a single shot it can produce several images refocused at different distances, depth maps, all-in-focus images and stereo pairs. A similar optical design can also be used in adaptive optics in astrophysics, in order to correct the aberrations caused by atmospheric turbulence in telescope images. In order to perform these tasks, different algorithms, running on GPU and FPGA, operate on the raw image captured by the camera.
Mitsubishi Electric Research Laboratories's (MERL) light-field camera is based on the principle of optical heterodyning and uses a printed film (mask) placed close to the sensor. Any hand-held camera can be converted into a light-field camera using this technology by simply inserting a low-cost film on top of the sensor. A mask-based design avoids the problem of loss of resolution, since a high-resolution photo can be generated for the focused parts of the scene.
Modern approaches to light field acquisition include mask-based devices that capture an optimized optical projection of a light field and reconstruct the high-resolution light field via compressive sensing.
Stanford University Computer Graphics Laboratory has developed a light-field microscope using a microlens array similar to the one used in the light-field camera developed by the lab. The prototype is built around a Nikon Eclipse transmitted light microscope/wide-field fluorescence microscope and standard CCD cameras. Light-field capturing ability is obtained by a module containing a microlens array and other optical components placed in the light path between the objective lens and camera, with the final multifocused image rendered using deconvolution. A later version of the prototype added a light-field illumination system consisting of a video projector (allowing computational control of illumination) and a second microlens array in the illumination light path of the microscope. The addition of a light-field illumination system both allowed for additional types of illumination (such as oblique illumination and quasi-dark-field) and correction for optical aberrations.
- E. H. Adelson and J. Y. A. Wang. Single Lens Stereo with Plenoptic Camera. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 14, No. 2, pp. 99-106, February 1992.
- R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan. Light Field Photography with a Hand-Held Plenoptic Camera. Stanford University Computer Science Tech Report CSTR 2005-02, April 2005.
- Lumsdaine, A., Georgiev, T., The Focused Plenoptic Camera, ICCP, April 2009.
- Ashok Veeraraghavan, Ramesh Raskar, Amit Agrawal, Ankit Mohan and Jack Tumblin. Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing. ACM Transactions on Graphics, Vol. 26, Issue 3, July 2007.
- Polydioptric Camera Design, says good for following moving objects.
- Computer scientists create a 'light field camera' that banishes fuzzy photos, Anne Strehlow. Stanford Report. November 3, 2005.
- One Camera With 40,000 Lenses Helps Prevent Blurry Images. Popular Science May 2011.
- The First Plenoptic Camera on the Market
- Lytro website
- Keats, Jonathon; Holland, Kris and McLeod, Gary. "PopSci's How It Works – 100 Megapixel Camera" (Adobe Flash). PopSci.com. Popular Science. Archived from the original on 2009-01-17. Retrieved 26 July 2009.
- Marwah, K.; Wetzstein, G.; Bando, Y.; Raskar, R. "Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections". ACM Transactions on Graphics (SIGGRAPH).
- A. Wang, P. Gill, and A. Molnar, "An angle-sensitive cmos imager for single-sensor 3d photography," in Solid-State Circuits Conference Digest of Technical Papers (ISSCC), 2011 IEEE International, Feb. 2011, pp. 412–414.
- "Pelican Imaging's 16-lens array camera coming to smartphones next year". May 2, 2013.
- Levoy M, Ng R, Adams A, Footer M, Horowitz M. 2006. Light Field Microscopy. ACM Transactions on Graphics 25(3):924–93. doi:10.1145/1141911.1141976.
- Levoy M, Zhang Z, McDowall I. 2009. Recording and controlling the 4D light field in a microscope. Journal of Microscopy. 235(2):144–162. doi:10.1111/j.1365-2818.2009.03195.x.
- Levoy M. 2008. Stanford Light Field Microscope Project (web page). Stanford Computer Graphics Laboratory.
- Article by Ren Ng of Stanford (now at Lytro)
- Say Sayonara to Blurry Pics. Wired.
- Fourier slice photography
- Light Field Microscopy video by Stanford Computer Graphics Laboratory.
- IEEE Spectrum article May 2012 Lightfield photography revolutionizes imaging, with sample images and diagrams of operation, retrieved 2012 May 11