ISPs don't censor the internet, with a few notable exceptions: since the Demon vs Godfrey case in 2001, ISPs have been legally obliged to remove allegedly defamatory material from their servers or face prosecution, and many mobile phone connections have adult content filters switched on by default to stop children accessing content they shouldn't.
The main exception is child pornography, which ISPs block by default. Websites are usually blocked in one of two ways: DNS filtering or blacklists. DNS filtering is the simplest method, but it can be a very blunt instrument.
When you block a specific site by its IP address, you also block any of its subdomains. This is how US authorities wrongly blocked some 84,000 subdomains last year, replacing their websites with notices claiming that they were involved in illegal pornography.
The other common approach, content filtering, is less likely to block legitimate websites. Instead of simply blocking IP addresses, content filtering checks requested URLs against a database of known sites.
If the URL isn't on the list, the page appears normally. If the URL is on the list, the traffic is routed to proxy servers that perform a more detailed check. If the URL passes this test, the requested page or file is shown. If it doesn't, the browser is redirected to a warning page or 'Page not found' message instead of the requested content.
That's how Cleanfeed, BT's illegal content filter, works. The blocklist comes courtesy of the Internet Watch Foundation, and it only checks for content that's been identified as child pornography. The IWF also monitors the internet for other material, including incitement to violence, hate speech and other obscene content, but that content isn't included on its URL blocklist.
All ISPs - including mobile phone data networks - have been required by law to use this kind of filtering since 2008. The big advantage of the Cleanfeed system is its precision. For example, it can block a single page or file and leave the rest of a website untouched.
Like any system, though, it isn't perfect: in 2008 it emerged that the IWF blocklist included a Wikipedia page about The Scorpions' album Virgin Killer, whose controversial artwork was deemed illegal under UK law. The blocklist censored the page describing the artwork, but not the artwork itself, and it temporarily prevented users of Cleanfeed-subscribing ISPs from updating any entries on Wikipedia whatsoever.
Mission creep
Despite the odd problem, few people would argue that filtering child pornography is a bad thing. However, any kind of site-blocking suffers from mission creep: if we can filter one type of content, politicians and pressure groups ask, why can't we filter another?
The most extreme example of such mission creep is in Australia. Its Labor Party wants ISPs to block any content that isn't suitable for under-15s, and which isn't protected by effective age verification systems, and its politicians have proposed that ISPs should also block euthanasia and anorexia websites, sites about illegal drugs, sites about abortion, overseas online gambling and so on.
Such mandatory filtering would be based on the ACMA (Australian Communications and Media Authority) blacklist, which, if the leaked version posted on Wikileaks is genuine, includes a dental surgery, a dog kennel and a travel agency. Could similar filtering happen here?
In 2006, Home Office minister Vernon Croaker admitted that the government had considered legislation forcing ISPs to block sites "glorifying terrorism" under the Terrorism Act 2006, while in December Ed Vaizey told the Sunday Times that ISPs should block all legal pornography by default "to protect children".
"I am hoping they will get their acts together so we don't have to legislate," Vaizey said. If you've ever run up against your phone provider's adult content filter while trying to access a perfectly reputable and non-pornographic website, you'll know that such filtering already happens and often tends to be overzealous.
In addition to filtering adult content, the government would like ISPs to filter another kind of content. In April, the culture minister met with ISPs to discuss the creation of an IWF-style blacklist - this time to filter sites accused of copyright infringement.
Secret blocks
"Website blocking is not straightforward," Trefor Davies says. "Consumer ISPs blocking the IWF list are only having to deal with around 600 URLs, updated twice daily. Having to gear up to cope with what are potentially hundreds of thousands of sites is another cost dimension again. The other issue is who takes responsibility for which sites to block. I wouldn't be happy as an ISP to take ownership, because I might be sued for wrongful blocking. I wouldn't be happy for rights holders to decide which sites are blocked. So it has to be a competent person like a judge, which introduces a whole new set of costs and issues."
The government is considering creating a third party to decide on such blocking. In March, it met ISPs and other interested parties to discuss creating an IWF-style organisation as a middleman between ISPs and rights holders, creating and curating a blocklist that ISPs would use to block copyright infringing websites.
Culture Secretary Jeremy Hunt has also asked OFCOM to review whether the site blocking measures outlined in the Digital Economy Act are realistic and practical. "I have no problem with the principle of blocking access to websites used exclusively for facilitating illegal download of content," Hunt said.
Not everybody is convinced that blocking copyright infringers is a good idea. Claims of copyright infringement have been used by the Church of Scientology to silence criticism, and by Amazon to justify booting WikiLeaks from its cloud computing servers, while the Recording Industry Association of America famously sued dead people and pensioners who didn't own computers in its crackdown on music file sharing. Who decides whether claims of infringement have merit?
As Peter Bradwell puts it: "We are concerned about extra-judicial censorship. On what grounds will these decisions be made? With what oversight? It's difficult to see how a working group featuring ISPs and rights holders can answer these questions."
Rather than secret lists, Bradwell predicts that: "Legitimate sites will inevitably be either placed on the list or caught accidentally." The ORG believes that the only effective and transparent way to deal with allegedly infringing content is to take the material off the web altogether. So is the ORG telling Ed Vaizey that?
"[In April], the working group met again without any civil society groups involved," Bradwell says. "We have requested that as well as the 'consumer representative' groups [Vaizey] mentions, he meets with rights groups concerned about these proposals. As well as the Open Rights Group, we suggested Index on Censorship, Global Partners, Consumer Focus and Article 19."
So far those suggestions haven't been embraced. "It's frustrating to continually have to make the case that these are broad public interest issues that can't be answered by a narrow group of stakeholders," Bradwell says. "Designing policy for such a group has only brought us bad policy, whether it's the cripplingly poor Digital Economy Act or the secretly negotiated international trade agreement ACTA."
Slippery slopes
The key argument against ISPs shaping what we see is that it's a slippery slope that leads inevitably to oppressive censorship - but the introduction of Cleanfeed hasn't infringed everyday users' activities, mobile phone networks' smut filters have no obvious downside apart from the odd wrongly flagged site, and the Digital Economy Act seems fairly toothless so far.
Are we worrying about nothing? "The slippery slope argument is one of the arguments," Bradwell says. "The force of that argument comes from a lack of clarity around a question that sits at the heart of this debate: who should decide what you are allowed to look at? We don't believe it's acceptable for private arrangements that bypass proper accountability and oversight, to have jurisdiction over your attention."
BT agrees. "Other than for the very particular and most heinous issue of child abuse images - on which BT, like most other ISPs, works with the IWF to prevent inadvertent access by its users - it isn't for ISPs to determine the rights or wrongs of sites related to other issues," Jarvis explains. "It's for the government and parliament to decide policy on site blocking, and in doing so they must consider the practicalities of implementation and the impact on the internet and its users."
We don't hold BT responsible for the things people say using its phone lines, we don't hold Royal Mail responsible for the contents of the letters it carries and, with a few exceptions, we don't hold ISPs responsible for the data they transmit. If that changes, are we losing something important?
"There is an argument that the previous decade will be remembered as a time of wild internet freedom, and that the future of the internet will be more restricted and more closely monitored," Bradwell says.
"The platforms and services [we use] are private organisations, whether they are ISPs or platforms like Twitter or Facebook." The internet is often compared to the Old 'Wild' West, but of course even the Old West was tamed - and the internet will be too.
The concern is that in their efforts to tame the net, politicians and private organisations could damage it. Calls to regulate our 21st century technology raise a first-century question: who watches the watchmen?
