ALIWEB (Archie Like Indexing for the WEB) is considered the first Web search engine, as its predecessors were either built with different purposes (the Wanderer, Gopher) or were literally just indexers (Archie, Veronica and Jughead).
First announced in November 1993 by developer Martijn Koster while working at Nexor, and presented in May 1994 at the First International Conference on the World Wide Web at CERN in Geneva, ALIWEB preceded WebCrawler by several months.
ALIWEB allowed users to submit the locations of index files on their sites which enabled the search engine to include webpages and add user-written page descriptions and keywords. This empowered webmasters to define the terms that would lead users to their pages, and also avoided setting bots (e.g. the Wanderer, JumpStation) which used up bandwidth. As relatively few people submitted their sites, ALIWEB was not very widely used.
Martijn Koster, who was also instrumental in the creation of the Robots Exclusion Standard, detailed the background and objectives of ALIWEB with an overview of its functions and framework in the paper he presented at CERN.
- Martijn Koster (30 November 1993). "ANNOUNCEMENT: ALIWEB (Archie-Like Indexing for the WEB)". comp.infosystems (plaintext version).
- "List of PostScript files for the WWW94 advance proceedings". First International Conference on the World-Wide Web. June 1994. "Title: "Aliweb - Archie-Like Indexing in the Web." Author: Martijn Koster. Institute: NEXOR Ltd., UK. PostScript, Size: 213616, Printed: 10 pages"
- Chris Sherman (3 December 2002). "Happy Birthday, Aliweb!". Search Engine Watch.
- Wes Sonnenreich (1997). "A History of Search Engines". John Wiley & Sons website.
- Martijn Koster. "Robots Exclusion". robotstxt.org.
- Martijn Koster. "Robots in the Web: threat or treat?". Reprinted with permission from ConneXions, The Interoperability Report, Volume 9, No. 4, April 1995.
- Martijn Koster. "Historical Web Services: ALIWEB". Martijn Koster's Historical Web Services page. "Note that I have nothing to do with aliweb.com. It appears some marketing company has taken the old aliweb code and data, and are using it as a site for advertising purposes. Their search results are worthless. Their claim to have trademarked "aliweb" I have been unable to confirm in patent searches. My recommendation is that you avoid them."