Distributed search engine
A distributed search engine is a search engine where there is no central server. Unlike traditional centralized search engines, work such as crawling, data mining, indexing, and query processing is distributed among several peers in a decentralized manner where there is no single point of control.
In April 2000 several programmers (including Gene Kan, Steve Waterhouse) built a prototype P2P web search engine based on Gnutella called InfraSearch. The technology was later acquired by Sun Microsystems and incorporated into the JXTA project. It was meant to run inside the participating websites' databases creating a P2P network that could be accessed through the InfraSearch website.
On May 31, 2000 Steelbridge Inc. announced development of OpenCOLA a collaborative distributive open source search engine. It runs on the user's computer and crawls the web pages and links the user puts in their opencola folder and shares resulting index over its P2P network.
QueenBee is a decentralized search engine project launched in August 2018 aiming to build fully decentralized search service on top of decentralized web. It shares advertising revenue to both content providers and peers that participate in indexing and ranking operations.
Dweb.page is an open source webpage that lets you share and search content on the distributed web based on IPFS. The search engine is running inside the webpage and uses a subscriber system to load relevant metadata.
- Justin Hibbard. "Can peer-to-peer grow up?". Red Herring.[permanent dead link]
- Simon Foust. "Move Over Yahoo, Here Comes InfraSearch". 'Dmusic. Archived from the original on 2000-10-13.
- Sean M. Dugan. "Peer-to-peer networking is poised to revolutionize the Internet once again". InfoWorld. Archived from the original on 2000-10-18.
- John Borland. "Napster-like technology takes Web search to new level". Cnet.
- David Akin. "Software launched with a little pop". Financial Post.[dead link]
- Paul Heltzel. "OpenCola-Have Some Code and a Smile". Technology Review.
- "YaCy: News". Archived from the original on 2005-11-24.
- Michael Christen. "Ich entwickle eine P2P-basierende Suchmaschine. Wer macht mit?". heise online.
Wolf Garbe. "BINGOOO - Die Transformation des World Wide Web zur virtuellen Datenbank" (in German). Wirtschaftinformatik (magazine).
... Wir setzen dem das Konzept einer verteilten Peer-to-Peer-Suchmaschine entgegen [We counter with the concept of a distributed peer-to-peer search engine] ...
Bernard Lunn. "Technical Q&A With FAROO Founder". ReadWriteWeb. Archived from the original on 2011-02-14.
... When I started to work on the first prototype in 2004 ...
- "FAROO: History". Archived from the original on 2008-03-22.
- "Revisited: Deriving crawler start points from visited pages by monitoring HTTP traffic". Faroo.
- Lai, Ziliang; Liu, Chris; Lo, Eric; Kao, Ben; Yiu, Siu-Ming (2018). "Decentralized Search on Decentralized Web". arXiv:1809.00939 [cs.IR].
- "Locking the Web Open: A Call for a Decentralized Web | Brewster Kahle's Blog". brewster.kahle.org. Retrieved 2018-09-30.
- Hawig, David. "A Distributed Search Engine for the Distributed Web". https://blog.florence.chat. Retrieved 13 December 2018. External link in