From Wikipedia, the free encyclopedia
Jump to: navigation, search

Dotbot is a web crawler developed by Dotmic.[1] As Dotmic an e-commerce search engine, Dotbot only examines e-commerce web sites. Unlike other crawlers, Dotbot can identify e-commerce websites. It is also capable of recognizing product name, image, prices, and description. The data collected from web pages by Dotbot are processed into information using various algorithms.

Dotbot is a crawler of[2] This crawling system is written in a mixture of C and Python.

Dotbot obeys the Robots exclusion standard,[3] the user-agent string of Dotbot is DotBot/1.1.[4]