Distributed search engine

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search

A distributed search engine is a search engine where there is no central server. Unlike traditional centralized search engines, work such as crawling, data mining, indexing, and query processing is distributed among several peers in a decentralized manner where there is no single point of control.



In April 2000 several programmers (including Gene Kan, Steve Waterhouse) built a prototype P2P web search engine based on Gnutella called InfraSearch. The technology was later acquired by Sun Microsystems and incorporated into the JXTA project.[1] It was meant to run inside the participating websites' databases creating a P2P network that could be accessed through the InfraSearch website.[2][3][4]


On May 31, 2000 Steelbridge Inc. announced development of OpenCOLA a collaborative distributive open source search engine.[5] It runs on the user's computer and crawls the web pages and links the user puts in their opencola folder and shares resulting index over its P2P network.[6]


On December 15, 2003 Michael Christen announced development of a P2P-based search engine, eventually named YaCy, on the heise online forums.[7][8]


In February 2001 Wolf Garbe published an idea of a peer-to-peer search engine,[9] started the Faroo prototype in 2004,[10] and released it in 2005.[11][12]



QueenBee is a decentralized search engine project launched in August 2018 aiming to build fully decentralized search service on top of decentralized web.[14] It shares advertising revenue to both content providers and peers that participate in indexing and ranking operations.

Dweb.page [15][edit]

Dweb.page is an open source webpage that lets you share and search content on the distributed web based on IPFS. The search engine is running inside the webpage and uses a subscriber system to load relevant metadata.

See also[edit]


  1. ^ Justin Hibbard. "Can peer-to-peer grow up?". Red Herring.[permanent dead link]
  2. ^ Simon Foust. "Move Over Yahoo, Here Comes InfraSearch". 'Dmusic. Archived from the original on 2000-10-13.
  3. ^ Sean M. Dugan. "Peer-to-peer networking is poised to revolutionize the Internet once again". InfoWorld. Archived from the original on 2000-10-18.
  4. ^ John Borland. "Napster-like technology takes Web search to new level". Cnet.
  5. ^ David Akin. "Software launched with a little pop". Financial Post.[dead link]
  6. ^ Paul Heltzel. "OpenCola-Have Some Code and a Smile". Technology Review.
  7. ^ "YaCy: News". Archived from the original on 2005-11-24.
  8. ^ Michael Christen. "Ich entwickle eine P2P-basierende Suchmaschine. Wer macht mit?". heise online.
  9. ^ Wolf Garbe. "BINGOOO - Die Transformation des World Wide Web zur virtuellen Datenbank" (in German). Wirtschaftinformatik (magazine). ... Wir setzen dem das Konzept einer verteilten Peer-to-Peer-Suchmaschine entgegen [We counter with the concept of a distributed peer-to-peer search engine] ...
  10. ^ Bernard Lunn. "Technical Q&A With FAROO Founder". ReadWriteWeb. Archived from the original on 2011-02-14. ... When I started to work on the first prototype in 2004 ...
  11. ^ "FAROO: History". Archived from the original on 2008-03-22.
  12. ^ "Revisited: Deriving crawler start points from visited pages by monitoring HTTP traffic". Faroo.
  13. ^ Lai, Ziliang; Liu, Chris; Lo, Eric; Kao, Ben; Yiu, Siu-Ming (2018). "Decentralized Search on Decentralized Web". arXiv:1809.00939 [cs.IR].
  14. ^ "Locking the Web Open: A Call for a Decentralized Web | Brewster Kahle's Blog". brewster.kahle.org. Retrieved 2018-09-30.
  15. ^ Hawig, David. "A Distributed Search Engine for the Distributed Web". https://blog.florence.chat. Retrieved 13 December 2018. External link in |website= (help)