Visual search engine

A Visual Search Engine is a search engine designed to search for information on the World Wide Web through the input of an image or a search engine with a visual display of the search results. Information may consist of web pages, locations, other images and other types of documents. This type of search engines is mostly used to search on the mobile Internet through an image of an unknown object (unknown search query). Examples are buildings in a foreign city. These search engines often use techniques for Content Based Image Retrieval.

A visual search engine searches images, patterns based on an algorithm which it could recognize and gives relative information based on the selective or apply pattern match technique.

Classification

Depending on the nature of the search engine there are two main groups, those which aim to find visual information and those with a visual display of results.

Visual information searchers

Screenshot of results shown by the image searcher through example GOS

Image search

An image search is a search engine that is designed to find an image. The search can be based on keywords, a picture, or a web link to a picture. The results depend on the search criterion, such as metadata, distribution of color, shape, etc., and the search technique which the browser uses.

Diagram of a search realized through example based on detectable regions from an image

Image search techniques

Two techniques currently used in image search:

Search by metadata: Image search is based on comparison of metadata associated with the image as keywords, text, etc.. and it is obtained a set of images sorted by relevance. The metadata associated with each image can reference the title of the image, format, color, etc.. and can be generated manually or automatically. This metadata generation process is called audiovisual indexing.

Search by example: In this technique, also called content-based image retrieval, the search results are obtained through the comparison between images using computer vision techniques. During the search it is examined the content of the image such as color, shape, texture or any visual information that can be extracted from the image. This system requires a higher computational complexity, but is more efficient and reliable than search by metadata.

There are image searchers that combine both search techniques, as the first search is done by entering a text, and then, from the images obtained can refine the search using as search parameters the images which appear as a result.

Video search

A video search is a search engine designed to search video on the net. Some video searchers process the search directly in the Internet, while others shelter the videos from which the search is done. Some searchers also enable to use as search parameters the format or the length of the video. Usually the results come with a miniature capture of the video.

Video search techniques

Currently, almost all video searchers are based on keywords (search by metadata) to perform searches. These keywords can be found in the title of the video, text accompanying the video or can be defined by the author. An example of this type of search is YouTube.

Some searchers generate keywords manually, while others use algorithms to analyze the audiovisual content of the video and to generate labels. The combination of these two processes improves the reliability of the search.

3D Models searcher

A searcher of 3D models aims to find the file of a 3D modeling object from a database or network. At first glance the implementation of this type of searchers may seem unnecessary, but due to the continuous documentary inflation of the Internet, every day it becomes more necessary indexing information.

3D Models search techniques

These have been used with traditional text-based searchers (keywords / tags), where the authors of the indexed material, or Internet users, have contributed these tags or keywords. Because it is not always effective, it has recently been investigated in the implementation of search engines that combine the search using text with the search compared to 2D drawings, 3D drawings and 3D models.

Princeton University has developed a search engine that combines all these parameters to perform the search, thus increasing the efficiency of search.

Imaginestics LLC created the world's first online shape search engine in the fall of 2005. They currently use VizSeek search engine technology in the industrial and manufacturing settings to help discover parts using shape as the matching criteria.

Mobile visual search

A mobile image searcher is a type of search engine designed exclusively for mobile phones, through which you can find any information on Internet, through an image made with the own mobile phone or using certain words (keywords).

Introduction

Mobile phones have evolved into powerful image and video processing devices equipped with high-resolution cameras, color displays, and hardware-accelerated graphics. They are also increasingly equipped with a global positioning system and connected to broadband wireless networks. All this enables a new class of applications that use the camera phone to initiate search queries about objects in visual proximity to the user (Figure 1). Such applications can be used, e.g., for identifying products, comparison shopping, finding information about movies, compact disks (CDs), real estate, print media, or artworks.

Process

Typically, this type of search engine uses techniques of query by example or Image query by example, which use the content, shape, texture and color of the image to compare them in a database and then deliver the approximate results from the query.

The process used in these searches in the mobile phones is as follows:

First, the image is sent to the server application. Already on the server, the image will be analyzed by different analytical teams, as each one is specialized in different fields that make up an image. Then, each team will decide if the submitted image contains the fields of their speciality or not.

Once this whole procedure is done, a central computer will analyze the data and create a page of the results sorted with the efficiency of each team, to eventually be sent to the mobile phone.

Applications

Google Goggles is the most popular application of image search engines, developed by Google Labs. Available for Android only today.

CamFind is currently themost accurate mobile visual search engine on the market. It's also available for both Android and iOS. It has gained over a million downloads on iOS alone in its first 6 months after the public release.

JustVisual.com (formerly known as 'Superfish') and its LikeThat showcase apps are API for developers to create their own visual-search mobile app.

Other companies in the image recognition space are the reverse image search-engines TinEye and Google's "search by image" feature of Google Images.

Visual display searchers

Another type of visual search is a search engine that shows results with a visual display image. This is an alternative to the traditional results of a sequence of links. Through some kind of image display, such as graphs, diagrams, previews of the websites, etc., it presents the results visually so that it is easier to find the desired material. Such search engines like Kiddle and Manzia present a new concept in the presentation of results, but the search techniques used are the same as in other search engines.