Content-based image retrieval

From Wikipedia, the free encyclopedia
Jump to: navigation, search
General scheme of content-based image retrieval

Content-based image retrieval (CBIR), also known as query by image content (QBIC) and content-based visual information retrieval (CBVIR) is the application of computer vision techniques to the image retrieval problem, that is, the problem of searching for digital images in large databases (see this survey[1] for a recent scientific overview of the CBIR field). Content based image retrieval is opposed to concept based approaches (see concept based image indexing).

"Content-based" means that the search will analyze the actual contents of the image rather than the metadata such as keywords, tags, and/or descriptions associated with the image. The term 'content' in this context might refer to colors, shapes, textures, or any other information that can be derived from the image itself. CBIR is desirable because most web based image search engines rely purely on metadata and this produces a lot of garbage in the results.[clarification needed] Also having humans manually enter keywords for images in a large database can be inefficient, expensive and may not capture every keyword that describes the image.[citation needed] Thus a system that can filter images based on their content would provide better indexing and return more accurate results.[citation needed]

Contents

History [edit]

The term content-based image retrieval (CBIR) seems to have originated in 1992, when it was used by T. Kato to describe experiments into automatic retrieval of images from a database, based on the colors and shapes present.[2] Since then, the term has been used to describe the process of retrieving desired images from a large collection on the basis of syntactical image features. The techniques, tools and algorithms that are used originate from fields such as statistics, pattern recognition, signal processing, and computer vision.

Technical progress [edit]

There is a growing interest in CBIR because of the limitations inherent in metadata-based systems, as well as the large range of possible uses for efficient image retrieval. Textual information about images can be easily searched using existing technology, but requires humans to personally describe every image in the database. This is impractical for very large databases, or for images that are generated automatically, e.g. from surveillance cameras. It is also possible to miss images that use different synonyms in their descriptions. Systems based on categorizing images in semantic classes like "cat" as a subclass of "animal" avoid this problem but still face the same scaling issues.

Potential uses for CBIR include:

CBIR software systems [edit]

See CBIR engines for other examples of publicly available and accessible CBIR systems.

CBIR techniques [edit]

Many CBIR systems have been developed, but the problem of retrieving images on the basis of their pixel content remains largely unsolved.

Query techniques [edit]

Different implementations of CBIR make use of different types of user queries.

Query by example is a query technique that involves providing the CBIR system with an example image that it will then base its search upon. The underlying search algorithms may vary depending on the application, but result images should all share common elements with the provided example.

Options for providing example images to the system include:

  • A preexisting image may be supplied by the user or chosen from a random set.
  • The user draws a rough approximation of the image they are looking for, for example with blobs of color or general shapes.[25]

This query technique removes the difficulties that can arise when trying to describe images with words.

Semantic retrieval [edit]

The ideal CBIR system from a user perspective would involve what is referred to as semantic retrieval, where the user makes a request like "find pictures of Abraham Lincoln". This type of open-ended task is very difficult for computers to perform - pictures of chihuahuas and Great Danes look very different, and Lincoln may not always be facing the camera or in the same pose. Current CBIR systems therefore generally make use of lower-level features like texture, color, and shape, although some systems take advantage of very common higher-level features like faces (see facial recognition system). Not every CBIR system is generic. Some systems are designed for a specific domain, e.g. shape matching can be used for finding parts inside a CAD-CAM database.

Other query methods [edit]

Other query methods include browsing for example images, navigating customized/hierarchical categories, querying by image region (rather than the entire image), querying by multiple example images, querying by visual sketch, querying by direct specification of image features, and multimodal queries (e.g. combining touch, voice, etc.)[26]

CBIR systems can also make use of relevance feedback, where the user progressively refines the search results by marking images in the results as "relevant", "not relevant", or "neutral" to the search query, then repeating the search with the new information.

Content comparison using image distance measures [edit]

The most common method for comparing two images in content based image retrieval (typically an example image and an image from the database) is using an image distance measure. An image distance measure compares the similarity of two images in various dimensions such as color, texture, shape, and others. For example a distance of 0 signifies an exact match with the query, with respect to the dimensions that were considered. As one may intuitively gather, a value greater than 0 indicates various degrees of similarities between the images. Search results then can be sorted based on their distance to the queried image.[25] A long list of distance measures can be found in.[27]

Color [edit]

Computing distance measures based on color similarity is achieved by computing a color histogram for each image that identifies the proportion of pixels within an image holding specific values (that humans express as colors). Current research is attempting to segment color proportion by region and by spatial relationship among several color regions. Examining images based on the colors they contain is one of the most widely used techniques because it does not depend on image size or orientation. Color searches will usually involve comparing color histograms, though this is not the only technique in practice.

Texture [edit]

Texture measures look for visual patterns in images and how they are spatially defined. Textures are represented by texels which are then placed into a number of sets, depending on how many textures are detected in the image. These sets not only define the texture, but also where in the image the texture is located.

Texture is a difficult concept to represent. The identification of specific textures in an image is achieved primarily by modeling texture as a two-dimensional gray level variation. The relative brightness of pairs of pixels is computed such that degree of contrast, regularity, coarseness and directionality may be estimated (Tamura, Mori & Yamawaki, 1978). However, the problem is in identifying patterns of co-pixel variation and associating them with particular classes of textures such as silky, or rough.

Shape [edit]

Shape does not refer to the shape of an image but to the shape of a particular region that is being sought out. Shapes will often be determined first applying segmentation or edge detection to an image. Other methods like [Tushabe and Wilkinson 2008] use shape filters to identify given shapes of an image. In some case accurate shape detection will require human intervention because methods like segmentation are very difficult to completely automate.

Applications [edit]

Some software producers are trying to push CBIR based applications into the Internet and law enforcement markets for the purpose of identifying and censoring images with skin-tones and shapes that could indicate the presence of nudity, with controversial results.

Zoomy Images, a stock photo agency launched in April 2013, enhanced its service by utilizing CBIR in its "Visually Similar Images" and "Reverse Image Search" functions, allowing clients to view more accurate search results versus its competitors similar images based on keywords.

See also [edit]

References [edit]

  1. ^ Content-based Multimedia Information Retrieval: State of the Art and Challenges, Michael Lew, et al., ACM Transactions on Multimedia Computing, Communications, and Applications, pp. 1–19, 2006.
  2. ^ Content-based Image Retrieval, John Eakins and Margaret Graham, University of Northumbria at Newcastle
  3. ^ "CIRES: Content based Image REtrieval System". Amazon.ece.utexas.edu. Retrieved 2012-10-18. 
  4. ^ "Vision Research Lab - Cortina: Large-Scale, Content-based Image Retrieval on the WWW". Vision.ece.ucsb.edu. Retrieved 2012-10-18. 
  5. ^ [1][dead link]
  6. ^ "Home". Kbkgroup.org. Retrieved 2012-10-18. 
  7. ^ "Imprezzeo". Imprezzeo. Retrieved 2012-10-18. 
  8. ^ "Server version - imgSeek". Server.imgseek.net. Retrieved 2012-10-18. 
  9. ^ Visual Information Retrieval using Java and LIRE, Mathias Lux, Oge Marques, Synthesis Lectures on Information Concepts, Retrieval, and Services 2013 5:1, 1-112
  10. ^ "LIRE". Lire Project. Retrieved 2013-02-28. 
  11. ^ The GRire library
  12. ^ [2][dead link]
  13. ^ "Octagon - content based image retrieval system". Octagon.viitala.eu. Retrieved 2012-10-18. 
  14. ^ "PIBE Home Page". Db.deis.unibo.it. Retrieved 2012-10-18. 
  15. ^ http://www.revimg.com RevIMG
  16. ^ SHIATSU (University of Bologna, Italy)
  17. ^ "Visual Search Technology - App Monetization - Search By Image". Superfish. 2011-12-05. Retrieved 2012-10-18. 
  18. ^ "Chic Engine : fashion search engine for the fashion 2.0 world". Chic Engine. 2013-02-16. Retrieved 2013-02-16. 
  19. ^ [3][dead link]
  20. ^ Yi Li. "demo: Fids". Cs.washington.edu. Retrieved 2012-10-18. 
  21. ^ "pixolution - vWeb-Demo". Pixolution. Retrieved 2012-10-18. 
  22. ^ "pixolution". Pixolution. Retrieved 2012-10-18. 
  23. ^ "The Visual Factory by XediX". Visual-recognition.com. Retrieved 2012-10-18. 
  24. ^ "Windsurf Home Page". Db.deis.unibo.it. Retrieved 2012-10-18. 
  25. ^ a b Shapiro, Linda; George Stockman (2001). Computer Vision. Upper Saddle River, NJ: Prentice Hall. ISBN 0-13-030796-3. 
  26. ^ Liam M. Mayron. "Image Retrieval Using Visual Attention". Mayron.net. Retrieved 2012-10-18. 
  27. ^ Eidenberger, Horst (2011). “Fundamental Media Understanding”, atpress. ISBN 978-3-8423-7917-6.

Bibliography [edit]

  • Bird, C.L.; P.J. Elliott, Griffiths (1996). User interfaces for content-based image retrieval. 
  • Rui, Yong; Thomas S. Huang, Shih-Fu Chang (1999). Image Retrieval: Current Techniques, Promising Directions, and Open Issues. 
  • Datta, Ritendra; Dhiraj Joshi, Jia Li, James Z. Wang (2008). "Image Retrieval: Ideas, Influences, and Trends of the New Age". ACM Computing Surveys 40 (2): 1–60. doi:10.1145/1348246.1348248. 
  • Tushabe, F.; M.H.F. Wilkinson (2008). "Content-based Image Retrieval Using Combined 2D Attribute Pattern Spectra". Springer Lecture Notes in Computer Science. 

Further reading [edit]

Relevant research papers

External links [edit]

Retrieved from "http://en.wikipedia.org/w/index.php?title=Content-based_image_retrieval&oldid=555466926"