Cleanfeed (content blocking system)

From Wikipedia, the free encyclopedia
Jump to: navigation, search

Cleanfeed is the name given to various privately administered ISP level content filtering systems operating in the United Kingdom and Canada, and as of May 2012 undergoing testing in Australia with a view to future mandatory implementation. These government-mandated programs originally attempted to block access to child pornography and abuse content located outside of the nation operating the filtering system. (Offending Web sites in the nation are breaking the law; the sites are shut down rather than being blocked.) The sites blocked are considered potentially illegal; actual legality can only be determined in a court of law. In the UK the Cleanfeed was temporarily extended in some named ISPs [1] until this process was moved to ISP-specific programmes. In October 2013, the UK government announced that as part of anti-terrorist measures it is considering a similar technology to block "extremist" material[2][3] [4]


United Kingdom[edit]

Cleanfeed is a content blocking system technology implemented in the UK by BT, Britain's largest Internet provider. It was created in 2003 and went live in June 2004.[5] BT spokesman Jon Carter described Cleanfeed's function as "to block access to illegal Web sites that are listed by the Internet Watch Foundation", and described it as essentially a server hosting a filter that checked requested URLs for Web sites on the IWF list, and returning an error message of "Web site not found" for positive matches.[6][7][8] Cleanfeed is a silent content filtering system, which means that Internet users cannot ascertain whether they are being regulated by Cleanfeed, facing connection failures, or the page really does not exist.

By the beginning of 2006 Cleanfeed was used by 80% of Internet service providers.[9] By the middle of 2006 the government reported that 90% of domestic broadband connections were either currently blocking or had plans to by the end of the year. Home Office minister Alan Campbell pledged that all ISPs would block access to child abuse websites by the end of 2007[10] and UK Home Office Minister Vernon Coaker instructed all UK ISPs to implement a version of Cleanfeed by the end of 2007 on a voluntary basis, or face legal compulsion.[11] However, no legislation was ever introduced and ISPs are still free to join on a voluntary basis.[12] Despite the target for 100% coverage being set for the end of 2007,[13] by the middle of 2008 the proportion of consumer broadband connections that were covered was only 95%.[14] In February 2009, the Government said that it is looking at ways to cover the final 5%.[15] A report in March 2014 by the Culture, Media and Sport Committee reported 98.6%[16] of domestic broadband lines are subject to blocking arrangements.

The Internet Watch Foundation used to[17] also take reports from the public and IT professionals until 2011. This content is not included in the IWF URL list supplied to the online industry for blocking purposes.

Cleanfeed targets only alleged child sexual abuse content identified by the Internet Watch Foundation. In June 2011 the Motion Picture Association began court proceedings in an attempt to force BT to use Cleanfeed to block access to NewzBin2, a site indexing downloads of copyrighted content.[18] BT was ordered to block access to the site in late July[1] and in a later clarification, BT was given two weeks to implement the block starting at the end of October.[19] The case ( Twentieth Century Fox Film Corp & Ors v British Telecommunications Plc [2011])[20] only compels BT's ISP division to implement the block on NewzBin, it remains outside of remit of the IWF URL list which is strictly limited to blocking sites which host child sexual abuse content.[21]

Technical implementation[edit]

There is a confidential url hash blacklist containing URLs of pages (not whole sites) to be blocked. A less confidential list of sites potentially containing blocked pages is available to ISPs. Routers on the edge redirect traffic to these sites to special HTTP proxy servers which perform the actual filtering by matching HTTP requests to URLs on the blacklist. Traffic that does not match the specific URL is forwarded through the proxy filter.

Chapter 7 of a research paper by Richard Clayton[22] provides an overview of the Cleanfeed technology.

The routers at an ISP that has implemented Cleanfeed technology check traffic destination against a list of IP addresses of sites that are suspected of hosting filtered traffic. If there is no match, the traffic is directed to the content host:

URL Filtering (1).svg

If the site IP address is found in the list of suspected sources of unwanted material, the traffic is routed to proxies (highlighted as IWF proxies)[23] that check the specific page against a confidential blacklist of pages.

URL Filtering (2).svg

This two-pass implementation reduces the load on the proxy servers by not requiring that all traffic pass through them.

Technical Detection[edit]

Due to the filtering mechanism, to detect whether a site is being filtered via an individual ISP's connection, one must first capture the filtering servers IPs by running traceroute for the first few hops to some websites known to be blocked[22][24]

In the example below, the IP addresses in bold are the ISP filtering servers.

Filtering comparison[edit]

The other popular way of blocking content is DNS manipulation. Compared to this, Cleanfeed has the following properties:

  • Slightly harder to circumvent, although users can use open proxies, or the Tor network, and servers can use another port than 80, or HTTPS.
  • Less collateral damage. DNS-based blocking is criticized for blocking all content on a site with the same domain name. Cleanfeed only blocks what is explicitly blacklisted. For example, it would be possible to block only one image in an article. DNS-based schemes also break the whole concept of DNS security.[25]

Related surveys of opinion[edit]

The first UK survey of Internet regulation was carried out in 2007 and 2008.[26][27] 90.21% of the participants in the limited scale survey were unaware of the existence of CleanFeed; of those who had heard about it, only 14.81% percent understood it completely. 11.1% learned about CleanFeed from UK government statements, and 22.2 percent from BT's statements. 60.87% did not trust BT, and 65.22% did not trust IWF to be responsible for a silent content blocking system in the UK.

A majority of the participants preferred an open content blocking system targeting child abuse content, rather than no Internet regulation. More specifically, 65.2% would prefer to see a message stating that a given site was blocked, 57.3% would like to have access to a form for unblocking a given site, and 68.5% would prefer more frequent briefing by BT, IWF and the UK.


Cleanfeed in Canada is a voluntary Internet URL filtering list maintained by for use by participating ISPs.[28] Eight major providers, representing approximately 80% of Canada's Internet users, have been using the list since November 2006 to block foreign websites.[29][30][31]

Accessing child pornography online is illegal in Canada. is recognized by the federal government, provincial governments and domestic law enforcement agencies as able to assess the potential illegality of alleged child pornography online. The Cleanfeed system is not a law enforcement tool and does not log access to URLs on the list.

Proposed implementations[edit]


Cleanfeed in Australia is a proposed mandatory ISP level content filtration system. It was proposed by the Kim Beazley led Australian Labor Party opposition in a 2006 press release, with the intention of protecting children who were vulnerable due to claimed parental computer illiteracy.[citation needed] It was announced on 31 December 2007 as a policy to be implemented by the Rudd ALP government, and initial tests in Tasmania produced a report in 2008. Public opposition and criticism quickly emerged, led by the EFA and gaining irregular mainstream media attention, with a majority of Australians reportedly "strongly against" its implementation.[32] Criticisms included expense, inaccuracy (it will be impossible to ensure only illegal sites are blocked) and the fact that it will be compulsory. Cleanfeed was quietly abandoned as a policy after the 2010 election.


One of the criticisms of Cleanfeed is its lack of transparency. This is a consequence of the list of blocked sites being secret. There are no safeguards to stop sites unrelated to Child Pornography being added to the list as a result of policy changes. It thus has a potential for the censorship of materials outside of its original remit. Indeed, the Home Office in the UK has previously indicated that it has considered requiring ISPs to block access to articles on the web deemed to be “glorifying terrorism”, within the meaning of the new Terrorism Act 2006, saying: "our legislation as drafted provides the flexibility to accommodate a change in Government policy should the need ever arise."[33] This has lead some to describe Cleanfeed as the most perfectly invisible censorship mechanism ever invented and to liken its powers of censorship to those employed currently by China.[34] However, at present no legislation is in place, and the implementation of the IWF URL list is still a voluntary agreement between ISPs and the IWF.

The measures have also been criticised for being inadequate as they only block accidental viewing and does not prevent content delivered through encrypted systems, file sharing, email and other systems.[35]

Another criticism is that Newzbin claims to have successfully circumvented Cleanfeed following a court order forcing BT to censor the website over copyright infringement claims.[36][37] This poses the question as to whether websites hosting child pornography could adopt similar measures to allow their users access to blocked content .

Due to the proxy server implementation of the Cleanfeed system, websites that filter users by IP address such as a wikis and file lockers will significantly broken through the system,[38] even if only a tiny proportion of its content is blocked.

Finally, information has surfaced that suggests that Cleanfeed could potentially be manipulated to provide a blacklist of blocked websites.[39][40] This is problematic as it could allow the dissemination of child pornography, rather than the prevention of access to it. Again this has led some to question Cleanfeed as a successful system for blocking illegal internet content.

See also[edit]


  1. ^ a b "BT ordered to block links to Newzbin 2 website". BBC News. 28 July 2011. 
  2. ^ "'s web filtering mission creep". The Register. 29 November 2013. 
  3. ^ "Hansard Oct 23rd 2013 - question 10". Hansard. 13 October 2013. 
  4. ^ "Ministers will order ISPs to block terrorist and extremist websites". The Guardian. 27 November 2013. 
  5. ^ Bright, Martin (6 June 2004). "BT puts block on child porn sites". The Guardian (London). Retrieved 30 April 2010. 
  6. ^ "How net providers stop child porn", BBC News, 7 February 2006. Retrieved 29 May 2006.
  7. ^ Arnfield, Robin (20 July 2004). "BT Technology Blocks Online Pornography". NewsFactor Network. 
  8. ^ "IWF/BT Project Cleanfeed", Internet Watch Foundation. Retrieved 29 May 2006.
  9. ^ Paul Goggins (Parliamentary Under-Secretary, Home Office) Commons, 13 February 2006 col. 1130 Internet (child pornography)
  10. ^ "Home Office clueless: The transcript". Computer Shopper. 17 March 2009. 
  11. ^
  12. ^ Internet Watch Foundation - "URL List". Accessed 12th November 2013
  13. ^ Vernon Coaker (Parliamentary Under-Secretary, Home Office) Written Answer, 15 May 2006 col. {{{column}}} Child Abuse (Internet)
  14. ^ Vernon Coaker (Parliamentary Under-Secretary, Home Office) Written Answer, 16 June 2008 col. {{{column}}} Pornography: Internet
  15. ^ "Online child abuse images warning". BBC News. 23 February 2009. Retrieved 5 May 2010. 
  16. ^ "Online safety - Page 16". Retrieved 19 March 2014. 
  17. ^ Incitement to racial hatred removed from IWF’s remit
  18. ^ "Film-makers seek injunction to block pirate site". BBC News. 28 June 2011. Retrieved 28 June 2011. 
  19. ^ "UK ISP BT Given 14 Days To Block Newzbin2". TorrentFreak. 26 October 2011. Retrieved 31 October 2011. 
  20. ^ [2012] 1 All ER 806, [2012] Bus LR 1471, [2011] RPC 28, [2011] EWHC 1981 (Ch)
  21. ^ Internet Watch Foundation. Blocking Good Practice. Accessed 12th November 2013
  22. ^ a b Clayton, Richard (November 2005). "Anonymity and traceability in cyberspace". Richard Clayton. Richard Clayton. Retrieved 10 December 2008. 
  23. ^ Hogge, Becky. "IWF censors Wikipedia, chaos ensues". Open Rights Group. Retrieved 2008-12-10. 
  24. ^ "Check The IWF list". 
  25. ^ Steve Crocker, Shinkuro, Inc., David Dagon, Georgia Tech, Dan Kaminsky, DKH, Danny McPherson, Verisign, Inc., Paul Vixie, Internet Systems Consortium (2011). "Security and Other Technical Concerns Raised by the DNS Filtering Requirements in the PROTECT IP Bill". 
  26. ^ FirstMonday: Internet regulation: The need for more transparent Internet filtering systems and improved measurement of public opinion on Internet filtering, N Kopumartiz and A Veglis
  27. ^ Book: BT's Cleanfeed and Online Censorship in UK, N Koumartzis, London College of Communication (University of the Arts London)
  28. ^ Cleanfeed Canada
  29. ^ ISPs and Tipline Step Up Battle Against Internet Child Exploitation
  30. ^ Mountie hopes web initiative could cut child abuse
  31. ^ ISPs to monitor child porn under proposed bill
  32. ^ Liebhardt, John (11 December 2008). "Australia rallies to "Stop the Clean Feed"". Global Voices Online. Retrieved 2008-12-11. 
  33. ^ "Government sets deadline for universal network-level content blocking", LINX, 29 May 2006. Retrieved 29 May 2006.
  34. ^ Professor Lilian Edwards, University of Southampton. "From child porn to China, in one Cleanfeed". 
  35. ^ "Restricting All but the Predators", Dark Reading, 14 June 2006. URL accessed on 24 June 2006.
  36. ^ Newzbin claims BT block 'not working'
  37. ^ Banned Piracy Website Expands BT Circumvention Tool to Include The Pirate Bay
  38. ^ Anonymous editing from your Internet Provider is disabled, please log in.
  39. ^ Failures in a Hybrid Content Blocking System
  40. ^ Back door to the black list

External links[edit]