Ad filtering or ad blocking is removing or altering advertising content in a webpage. Advertising can exist in a variety of forms including pictures, animations, embedded audio and video, text, or pop-up windows and can employ autoplay of audio and video. It is a known problem with most web browsers, including Firefox, that restoring sessions often plays multiple embedded ads at once. All browsers offer some solution to the problem, either by targeting technologies (Adobe Flash/Shockwave, Windows Media Audio files, etc.) that are used to deliver ads, targeting URLs that are the source of ads, or targeting behavioural characteristic of ads (such as the use of HTML5 autoplay of both audio and video).
- 1 Benefits
- 2 Methods
- 3 Limitations
- 4 Economic consequences for online business
- 5 Advertiser offensive countermeasures and justifications
- 6 Advertiser network blocking
- 7 See also
- 8 References
To users, the benefits of ad blocking include quicker loading and cleaner looking Web pages free from advertisements, lower resource waste (bandwidth, CPU, memory, etc.), and privacy benefits gained through the exclusion of the tracking and profiling systems of ad delivery platforms. Blocking ads can also save substantial amounts of energy.
Another important aspect is improving security; according to some research, online advertising subjects users to a higher risk of infecting their devices than surfing porn sites. In a high-profile case malware was distributed through advertisements provided to YouTube by a malicious customer of Google's Doubleclick.
Users who pay for total transferred bandwidth ("capped" or pay-for-usage connections) including most mobile users worldwide, have a direct financial benefit from filtering an ad before it is loaded. Streaming audio and video, even if they are not presented to the user interface, can rapidly consume gigabytes of transfer especially on a faster 4G connection. In Canada, where users without a data plan often pay C$0.50/megabyte ($500/gigabyte) for at least the first 50-100MB exceeding their data allowance, the cost of tolerating ads can be quite high. Even fixed connections are often subject to usage limits, especially the faster connections (100Mbit/s and up) which can quickly saturate a network if filled by streaming media.
One method of filtering is simply to block (or prevent autoplay of) Flash animation or image loading or Windows audio and video files. This can be done in most browsers easily and also improves security and privacy. This crude technological method is refined by numerous browser extensions. Every internet browser handles this task differently, but, in general, one alters the options, preferences or application extensions to filter specific media types. An additional add-on is usually required to differentiate between ads and non-ads using the same technology, or between wanted and unwanted ads or behaviors.
The more advanced filters allow fine-grained control of advertisements through features such as blacklists, whitelists, and regular expression filters. Certain security features also have the effect of disabling some ads. Some antivirus software can act as an ad blocker.
Filtering by intermediaries such as providers or national governments is increasingly common. See below especially re provider ad substitution and national root DNS.
Content filtering can be added to Firefox, Chromium-based browsers, Opera, Safari and other browsers with extensions such as AdBlock, Adblock Plus and uBlock, and a number of sources provide regularly updated filter lists. Adblock Plus is included on Maxthon by default.
A number of external applications offer ad filtering as a primary or additional feature. A traditional solution is to customize an HTTP proxy (or web proxy) to filter content. These programs work by caching and filtering content before it is displayed in a user's browser. This provides an opportunity to remove not only ads but also content which may be offensive, inappropriate, or simply junk. Popular proxy software which blocks content effectively include Netnanny, Privoxy, Squid, and some content-control software. The main advantage of the method is freedom from implementation limitations (browser, working techniques) and centralization of control (the proxy can be used by many users).
Proxies are very good at filtering but have several limitations compared to browser based solutions:
- Difficulty to filter SSL (https://) traffic
- Full webpage context is not available to the filter
Hosts file and DNS manipulation
Most operating systems, even those which are aware of the Domain Name System (DNS), still offer backwards compatibility with a locally administered list of foreign hosts. This configuration, for historical reasons, is stored in a flat text file that by default contains very few hostnames and their associated IP addresses. Editing this hosts file is simple and effective because most DNS clients will read the local hosts file before querying a remote DNS server. Storing blackhole entries in the hosts file prevents the browser from accessing an ad server by manipulating the name resolution of the ad server to a local or nonexistent IP address (127.0.0.1 or 0.0.0.0 are typically used for IPv4 addresses). While simple to implement, these methods are also very easy to circumvent. One method to circumvent this form of ad filtering is to load ads from servers with hard coded IP addresses, thus skipping name resolution altogether. Another method to evade this form of filtering is to load the advertisements from a server which also serves the main content; blocking name resolution of this server would also block the useful content of the site.
- Further information: hosts file.
Using a DNS sinkhole by manipulating the hosts file exploits the fact that most operating systems store a file with IP address, domain name pairs which is consulted by most browsers before using a DNS server to look up a domain name. By assigning the loopback address to each known ad server, the user directs traffic intended to reach each ad server to the local machine or to a virtual blackhole of /dev/null or bit bucket.
This method operates by filtering and changing records of a DNS cache. On most operating systems the domain name resolution always goes via DNS cache. By changing records within the cache or preventing records from entering the cache, programs are allowed or prevented from accessing domain names. The external programs monitor internal DNS cache and import DNS records from a file. As a part of the domain name resolution process, a DNS cache lookup is performed before contacting a DNS server. Thus its records take precedence over DNS server queries. Unlike the method of modifying a Hosts file, this method is more flexible as it uses more comprehensive data available from DNS cache records.
Advertising can be blocked by using a DNS server which is configured to block access to domains or hostnames which are known to serve ads by spoofing the address. Users can choose to use an already modified DNS server or set up a dedicated device such as a Raspberry Pi themselves.
Manipulating DNS is a widely employed method to manipulate what the end user sees from the internet but can also be deployed locally for personal purposes.
China runs its own root DNS and the EU has considered the same. Google has required their Google Public DNS be used for some applications on its Android devices. Accordingly, DNS addresses / domains used for advertising may be extremely vulnerable to a broad form of ad substitution whereby a domain that serves ads is entirely swapped out with one serving more local ads to some subset of users. This is especially likely in countries, notably Russia, India and China, where advertisers often refuse to pay for clicks or page views. DNS-level blocking of domains for non-commercial reasons is already common in China.
Devices such as AdTrap use their own hardware which one connects to their network to block internet advertising.
By external parties and internet providers
Internet providers, especially mobile operators, frequently offer proxies designed to reduce network traffic. Even when not targeted specifically at ad-filtering, these proxy-based arrangements will block many types of advertisements that are too large or bandwidth-consuming, or that are otherwise deemed unsuited for the specific internet connection or target device.
Many internet operators block some form of advertisement and inject their own.
As with most content filters, blocking advertisements presents the possibility that desirable content will be blocked in error, called a false positive.
Economic consequences for online business
One consequence of widespread ad blocking is decreased revenue to a website sustained by advertisements, where this blocking can be detected. An ad that is displayed but ignored does not provide value. But sites are rarely paid “per click”, instead the standard metric for selling web display ads is “CPM”, or “cost per mille” - the cost for a thousand views. Even if users ignore adverts, advertisers perceive a value in putting brands in front of them. In dollar terms, it would be typical to see a general-interest website lose 10%-15% of its revenue. For a website that serves 10 million page views per day, with three ads per page, sold at a rate of $5 CPM, ad blocking could translate into daily losses of about $20,000.
Advertiser offensive countermeasures and justifications
Some websites have taken counter-measures against ad-blocking software, such as attempting to detect the presence of ad blockers and informing users of their views, or outright preventing users from accessing the content unless they disable the ad-blocking software. There have been several arguments supporting and opposing the assertion that blocking ads is wrong.
Advertiser network blocking
The PIPCU maintains an 'Infringing Website List' (IWL), a portal for digital advertisers to be informed of sites containing infringing content with the intention that they cease advertising on them. Sites are identified as infringing by rights holders for the and the list is not made available to the public. As of August 12, a freedom of information request from TorrentFreak revealed: 74 domains are subject to the advertiser blocking programme, of which of October 2014 only 2 domains had ever been removed from the list. 83 advertising companies with a UK presence are currently participating.
Working with the media and advertiser industry body, the Audit Bureau of Circulations (ABC) created a technology portal called 'Project Sunblock'. If the PIPCU do not receive a response from the website operators, the host or registrar of an allegedly infringing site, the site is added to the IWL via the Sunblock portal, which is then passed along to participating advertising networks for blacklisting. From June 2014 this technology allowed replacing the adverts of websites believed to be offering pirate content with warnings from the PIPCU.
This is an ongoing investigation and disclosure to the public domain would raise the profile of those sites unlawfully providing copyright material. This would enable individuals to visit the sites highlighted and unlawfully download copyright material and increase the scale of the loss. In the case of advertisers, public identification would increase the risk of harm to them by way of cyber attack or other means.—City of London Police
- "Upon startup multiple audio sources begin playing. I can't find the tab to kill them! | Firefox Support Forum | Mozilla Support". Support.mozilla.org. Retrieved 2013-10-26.
- "Internet Security". SecTheory. 2008-10-24. Retrieved 2013-10-26.
- "Online Advertising More Likely to Spread Malware Than Porn". PCMAG.
- "YouTube angeblich als Virenschleuder missbraucht". heise.de.
- "The Wild Wild Web: YouTube ads serving malware". Bromium Labs.
- "Konqueror browser features". Retrieved 17 February 2015.
- "Use Tracking Protection in Internet Explorer". Retrieved 17 February 2015.
- "Adblock Plus integrated into Maxthon browser". Retrieved 17 February 2015.
- "Verigio Communications - Portable DNS Cache and Firewall for Windows". Verigio.com. Retrieved 2013-10-26.
- "A Simple DNS-Based Approach for Blocking Web Advertising". Deer Run Associates. 2013-08-13. Retrieved 2014-05-09.
- "Block ads on your iPad using a Raspberry Pi". MacGeeks.co.uk. 2014-06-15. Retrieved 2014-06-15.
- "The Extent of DNS Services Being Blocked in China". Circleid.com. 2010-05-21. Retrieved 2013-10-26.
- "AdTrap - The internet is yours again". BluePointSecurity.com. 2014-08-18. Retrieved 2014-08-18.
- Fisher, Ken (2010-03-06). "Why Ad Blocking is devastating to the sites you love". Ars Technica. Retrieved 2013-10-26.
- "Ad Blocking is Immoral | The Google Cache: Search Engine Marketing, SEO & PPC". The Google Cache. 2007-08-02. Retrieved 2011-11-04.
- "Adblock: Adapt, or die. Service Assurance Daily: Anything and everything that affects IT performance, from the mundane to the bizarre - Network Performance Blog". Networkperformancedaily.com. 2007-09-05. Retrieved 2011-11-04.
- Kirk, Jeremy (2007-08-23). "Firefox ad-blocker extension causes angst | Applications". InfoWorld. Retrieved 2011-11-04.
- City of London Police call on advertising and brand sectors to help tackle cyber crime, PIPCU, 2014-03-31
- UK Police Launch Pirate Site Blacklist for Advertisers, TorrentFreak, 2014-03-31
- Ernesto. "UK Police Hijack Ads of 74 Pirate Websites, Refuse to Name Them". TorrentFreak. Retrieved 21 September 2014.
- "Number of sites removed from the PIPCU Infringing Website List". Retrieved 13 December 2014.
- "Online Pirates Are Making Advertisers Walk the Plank". Retrieved 29 July 2014.
- Kobie, Nicole. "Policing the web: anti-piracy and beyond". http://www.pcpro.co.uk/. Retrieved 27 November 2014.
- Lee, Dave. "Police placing anti-piracy warning ads on illegal sites". Retrieved 29 July 2014.
- Cellan-Jones, Rory. "Web creator rejects net tracking". http://news.bbc.co.uk/. Retrieved 10 September 2014.
- "Infringing Website List". https://www.whatdotheyknow.com/. Retrieved 10 September 2014.