Pyramid (image processing)
Output of a typical corner detection algorithm
|Affine invariant feature detection|
Pyramid or 'pyramid representation' is a type of multi-scale signal representation developed by the computer vision, image processing and signal processing communities, in which a signal or an image is subject to repeated smoothing and subsampling. Historically, pyramid representation is a predecessor to scale-space representation and multiresolution analysis.
There are two main types of pyramids; lowpass pyramids and bandpass pyramids. A lowpass pyramid is generated by first smoothing the image with an appropriate smoothing filter and then subsampling the smoothed image, usually by a factor of two along each coordinate direction. This smoothed image is then subjected to the same processing, resulting in a yet smaller image. As this process proceeds, the result will be a set of gradually more smoothed images, where in addition the spatial sampling density decreases level by level. If illustrated graphically, this multi-scale representation will look like a pyramid, from which the name has been obtained. A bandpass pyramid is obtained by forming the difference between adjacent levels in a pyramid, where in addition some kind of interpolation is performed between representations at adjacent levels of resolution, to enable the computation of pixelwise differences.
Pyramid generation kernels
A variety of different smoothing kernels have been proposed for generating pyramids. Among the suggestions that have been given, the binomial kernels arising from the binomial coefficients stand out as a particularly useful and theoretically well-founded class. Thus, given a two-dimensional image, we may apply the (normalized) binomial filter (1/4, 1/2, 1/4) typically twice or more along each spatial dimension and then subsample the image by a factor of two. This operation may then proceed as many times as desired, leading to a compact and efficient multi-scale representation. If motivatived by specific requirements, intermediate scale levels may also be generated where the subsampling stage is sometimes left out, leading to an oversampled or hybrid pyramid. With the increasing computational efficiency of CPUs available today, it is in some situations also feasible to use wider support Gaussian filters as smoothing kernels in the pyramid generation steps.
Applications of pyramids
In the early days of computer vision, pyramids were used as the main type of multi-scale representation for computing multi-scale image features from real-world image data. More recent techniques include scale-space representation, which has been popular among some researchers due to its theoretical foundation, the ability to decouple the subsampling stage from the multi-scale representation, the more powerful tools for theoretical analysis as well as the ability to compute a representation at any desired scale, thus avoiding the algorithmic problems of relating image representations at different resolution. Nevertheless, pyramids are still frequently used for expressing computationally efficient approximations to scale-space representation.
Laplacian image pyramids based off the bilateral filter provide a good framework for image detail enhancement and manipulation. The difference images between each layer are modified to exaggerate or reduce details at different scales in an image.
- Burt, P. J. (May 1981). "Fast filter transform for image processing". Computer Graphics and Image Processing 16: 20–51. doi:10.1016/0146-664X(81)90092-7.
- Crowley, James L. (November 1981). A representation for visual information. Carnegie-Mellon University, Robotics Institute. tech. report CMU-RI-TR-82-07.
- Burt, Peter and Adelson, Ted, "The Laplacian Pyramid as a Compact Image Code", IEEE Trans. Communications, 9:4, 532–540, 1983.
- Crowley, J. L.; Parker, A. C. (March 1984). "A representation for shape based on peaks and ridges in the difference of low-pass transform". IEEE Transactions on Pattern Analysis and Machine Intelligence 6 (2): 156–170. doi:10.1109/TPAMI.1984.4767500. PMID 21869180.
- Crowley, J. L. and Sanderson, A. C. "Multiple resolution representation and probabilistic matching of 2-D gray-scale shape", IEEE Transactions on Pattern Analysis and Machine Intelligence, 9(1), pp 113-121, 1987.
- P. Meer, E. S. Baugher and A. Rosenfeld "Frequency domain analysis and synthesis of image generating kernels", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol 9, pages 512-522, 1987.
- Lindeberg, Tony, "Scale-space for discrete signals," PAMI(12), No. 3, March 1990, pp. 234-254.
- Lindeberg, Tony. Scale-Space Theory in Computer Vision, Kluwer Academic Publishers, 1994, ISBN 0-7923-9418-6
- See the article on multi-scale approaches for a very brief theoretical statement
- Crowley, J, Riff O. Fast computation of scale normalised Gaussian receptive fields, Proc. Scale-Space'03, Isle of Skye, Scotland, Springer Lecture Notes in Computer Science, volume 2695, 2003.
- Lindeberg, T. and Bretzner, L. Real-time scale selection in hybrid multi-scale representations, Proc. Scale-Space'03, Isle of Skye, Scotland, Springer Lecture Notes in Computer Science, volume 2695, pages 148-163, 2003.
- Lowe, D. G. (2004). "Distinctive image features from scale-invariant keypoints". International Journal of Computer Vision 60 (2): 91–110.
- Photo Detail Manipulation via Image Pyramids