An Internet bot, also known as web robot, WWW robot or simply bot, is a software application that runs automated tasks over the Internet. Typically, bots perform tasks that are both simple and structurally repetitive, at a much higher rate than would be possible for a human alone. The largest use of bots is in web spidering, in which an automated script fetches, analyses and files information from web servers at many times the speed of a human. Each server can have a file called
robots.txt, containing rules for the spidering of that server that the bot is supposed to obey or be removed.
In addition to their uses outlined above, bots may also be implemented where a response speed faster than that of humans is required (e.g., gaming bots and auction-site robots) or less commonly in situations where the emulation of human activity is required, for example chat bots.
IM and IRC
Some bots communicate with other users of Internet-based services, via instant messaging (IM), Internet Relay Chat (IRC), or another web interface. These chatterbots may allow people to ask questions in plain English and then formulate a proper response. These bots can often handle many tasks, including reporting weather, zip-code information, sports scores, converting currency or other units, etc. Others are used for entertainment, such as SmarterChild on AOL Instant Messenger and MSN Messenger.
An additional role of IRC bots may be to lurk in the background of a conversation channel, commenting on certain phrases uttered by the participants (based on pattern matching). This is sometimes used as a help service for new users, or for censorship of profanity.
AOL Instant Messenger has now introduced a feature that allows you to make a screen name into a bot. This new feature removes the rate limit on the screen name, however it is now limited in the amount of instant messages that can be sent and received.
There has been a great deal of controversy about the use of bots in an automated trading function. Auction website eBay has been to court in an attempt to suppress a third-party company from using bots to traverse their site looking for bargains; this approach backfired on eBay and attracted the attention of further bots. The United Kingdom-based bet exchange Betfair saw such a large amount of traffic coming from bots they launched a WebService API aimed at bot programmers through which Betfair can actively manage bot interactions.
Another, more malicious use of bots is the coordination and operation of an automated attack on networked computers, such as a denial-of-service attack by a botnet. Internet bots can also be used to commit click fraud and more recently have seen usage around MMORPG games as computer game bots. A spambot is an internet bot that attempts to spam large amounts of content on the Internet, usually adding advertising links.
- There are malicious bots (and botnets) of the following types:
- Spambots that harvest email addresses from contact or guestbook pages
- Downloader programs that suck bandwidth by downloading entire web sites
- Website scrapers that grab the content of websites and re-use it without permission on automatically generated doorway pages
- Viruses and worms
- DDoS attacks
- Botnets, zombie computers, etc.
- Bots are also used to buy up good seats for concerts, particularly by ticket brokers who resell the tickets. Bots are employed against entertainment event-ticketing sites, like TicketMaster.com. The bots are used by ticket brokers to unfairly obtain the best seats for themselves while depriving the general public from also having a chance to obtain the good seats. The bot runs through the purchase process and obtains better seats by pulling as many seats back as it can.
- Bots are often used in Massively Multiplayer Online Roleplaying Games to farm for resources that would otherwise take significant time or effort to obtain; this is a concern for most online in-game economies.
- Bots are also used to artificially increase views for YouTube videos.
- Bots are used to increase traffic counts on analytics reporting to extract money from advertisers. A study by comScore found that 54 percent of display ads shown in thousands of campaigns between May 2012 and February 2013 never appeared in front of a human being.
- in 2012 reporter Percy Lipinski reported that he discovered millions of bot or botted or pinged views at CNN iReport. CNN iReport quietly removed millions of views from the account of so-called superstar iReporter Chris Morrow. A followup investigation lead to a story published on the citizen journalist platform, Allvoices: http://www.allvoices.com/contributed-news/14694943-cnn-hit-hard-in-ongoing-pay-per-view-scandal-millions-of-page-view-removed. It is not known if the ad revenue received by CNN from the fake views was ever returned to the advertisers.
The most widely used anti-bot technique is the use of CAPTCHA, which is a form of Turing test used to distinguish between a human user and a less-sophisticated AI-powered bot, by the use of graphically encoded human-readable text. Examples of providers include Recaptcha, and commercial companies such as Minteye, Solve Media and NuCaptcha. Captchas, however, are not foolproof in preventing bots as they can often be circumvented by computer character recognition, security holes, and even by outsourcing captcha solving to cheap laborers.
Bots in popular culture
- The Basshunter song Boten Anna makes many references to IRC Bots, but confusingly also features the singer riding a boat.
- Individual bots
- Other bot-related links
- XBot Web Automation
- Artificial Solutions
- Server and SDK for conversational bot development
- Advanced Robots.txt Generator
- Cyber-Surveillance and Internet Data-Mining
- UBot Studio.com UBot Studio bot creation suite
- Web Bot: What is it? Can it predict stuff?
- WEB Robotica - Marketplace for WEB Bots like scrapers, spiders, crawlers and webbots. Used for extracting database from the World Wide Web and processing data from any website
- Robot journalism pushes the boundaries for what’s possible