DotBot

From Wikipedia, the free encyclopedia
Jump to: navigation, search

Dotbot is a web crawler developed by Dotmic.[1] Dotmic being an e commerce search engine, Dotbot only crawl e commerce web sites.Unlike other crawlers Dotbot can identify e commerce websites.It is also capable of recognizing product name,image,prices,description etc.The data collected from web page by dotbot are processed into information using various algorithms.

Dotbot is a crawler of Dotnetdotcom.org.[2] This crawling system is written in a mixture of C and Python.

Dotbot obeys the Robot Exclusion standard,[3] the user-agent string of Dotbot is DotBot/1.1.[4]

References[edit]