Database search engine
||This article may require copy editing for grammar, style, cohesion, tone, or spelling. (June 2015)|
Categories of search engine software include:
- Web search or full-text search (e.g. Lucene).
- Database or structured data search (e.g. Dieselpoint).
- Mixed or enterprise search (e.g. Google Search Appliance).
The largest online directories, such as Google and Yahoo, utilize thousands of computers to process billions of website documents using web crawlers or spiders, then return results for thousands of searches per second. Processing high volumes of queries requires software to run in a highly distributed environment with a high degree of redundancy.
Modern search engine components
Searching for a text-based content in databases or structured data formats (such as XML and CSV) presents special challenges and opportunities which specialized search engines resolve. Databases are slow when solving complex queries which have multiple logical or string matching arguments. However, databases allow logical queries such as the use of multi-field Boolean logic, while full-text searches do not. "Crawling" (a human by-eye search) is not necessary to find information stored in a database because the data is already structured, though it is often necessary to index the data in a more compact form to allow for faster searches.
Database search engines are usually included with major database software products. As such, they are commonly called "indexing engines." However, these indexing engines are relatively limited in their ability to customize indexing formats (such as compounding, normalization, transformation and translation). Usually, these search engines do not provide sophisticated data-matching technology (such as string matching, Boolean logic, algorithmic methods and search scripting).
More advanced database search systems, relational databases are indexed by compounding multiple tables into a single table containing only the fields that need to be "queried" (displayed in search results). The actual data-matching engines can include a multitude of functions including basic string matching, normalization and transformation. Database search technology is used heavily by many large public and private entities including government database services, e-commerce companies, online advertising platforms, telecommunications service providers and other consumers with a need to access information in large repositories.
Databases are the core of any software application or system and may get exponentially larger over time. Hence the need for a search engine to help manage the contents of large databases.