'People lie, internet stats tell the truth'

Tancer spends much of his day seeking patterns in a sample of 25 million taken worldwide by HitWise through ISPs, although he hastily points out that the data is anonymised and collated. Hitwise simply doesn't have any interest in tracking individuals.

TechRadar asks if it's sometimes easy to start seeing patterns where there are none, or add two and two together and make five.

"One thing I've written about since finishing the book is what I call narrative fallacy – so when you have a massive dataset you might fall in love with your hypothesis and really look at the data that supports it.

"So there is a little bit of intellectual honesty needed, and you have to be pretty diligent.

Lipstick on a pitbull

"I was writing a piece for Time and there's a theory in the states which as been propagated by the Chairman of Estee Lauder that in tough economic times sales of lipstick goes up.

"The theory is that lipstick is an affordable luxury – when times are tough it doesn't cost much to buy lipstick.

"So I charted the search term lipstick over a three year period to see if I could find a correlation and get this flat line up until September 2008 and then a massive increase in the searches on lipstick.

"I then superimposed the Dow Jones Index over the top and got that perfect negative correlation. I'm so excited; I write my column and I'm about to submit and then I decided to take a step back and looked a little deeper.

"One thing I can do with our data is to look at all the search terms that contain the work lipstick and when I did that the entire column was shot because the number one search term was 'lipstick on a pitbull'.

"The whole discussion between Sarah Palin and Barack Obama and that one phase had been a huge topic of conversation; people were looking for videos or articles. It just happened to coincide perfectly with a drop in the Dow."