Google Search is broken

Google's latest Chrome plugin isn't just a bit of technology: it's a tacit admission of failure.

The extension enables you to block certain kinds of sites, dubbed Content Farms, from appearing in the search results.

It's an admission of failure, because if Google did what it's supposed to do - that is, find the best, most relevant content - then you wouldn't need to block anything.

The problem is that Google's success has created a new kind of industry. Content farms are firms who produce what Google's Matt Cuts calls "shallow or low-quality content".

Cutts is a funny guy, and his screenshots show the parody site The Content Farm, but the point is a serious one: often, when you search Google for something, you don't necessarily get content that's been created for you; you get content that's been created for Google's search algorithm.

There's nothing unethical or illegal about content farms - they're not wicked, or dishonest, or evil - but their prominence in Google results means they can be an enormous pain in the backside.

Searching for how-tos often results in instructions that are so vague they're ridiculous - The Content Farm's spoof How To Pay Off Your Credit Card Debt article nails it with its first step: "Get money. You can acquire money by getting a job that pays money or by selling some stuff or maybe by robbing someone with money" - and it's particularly irritating if you're looking for something sensible, such as information about treating a medical condition.

Fighting the farms

That's the background. So what will the extension actually do? As Matt Cutts explains: "when you block a site with the extension, you won't see the results from that domain again in your Google search results."

You can change your mind if you block something by accident. From time to time Google will analyse that data and see what tweaks, if any, it needs to make to its algorithms.

It's bad news for firms such as Demand Media, whose billion-dollar valuations are based almost entirely on their sites' ability to score highly on Google search results, but will it make Google search more useful?

In the short term, the answer is probably yes - but when even reputable retailers are allegedly gaming Google, it's clear that there's a bigger issue than a few sites custom-tooling content.

The problem is that Google's power corrupts: scoring highly in Google Search is too lucrative for firms to simply stick stuff online and hope Google finds it. The hard bit for Google is working out what to do about it.

Content farms exist because Google didn't think their content was spam; classify it as spam, and two things will happen. Some sites will be excluded unfairly and will scream bloody murder; others will look for - and find - new techniques that Google doesn't currently see as spam. Eventually that'll become a problem and the whole dance begins again.

Part of the problem is that Google has a conflict of interest here: the content farms depend on ads, and most of those ads come from one company. The next time you land on a page written for PageRank instead of people, move your mouse over one of the ads and see which firm's URL turns up in the status bar. More often than not, it starts with a G.