TechRadar recently spoke to Vice President of OneRiot, the real-time search engine that aims to keep you up to date with exactly what's hot on the web at the moment. We wanted to find out why it's getting so much attention at the moment.
Real-time search is the ultimate buzzword on the web at the moment, with Twitter and Facebook just a few of the sites opening up the social possibilities of up-to-the-minute information, and with OneRiot taking this one step further, there's no time like the present to speak about the machinations behind the site, and just how it keeps right up to date…
TechRadar: Tell us a little bit about the history of OneRiot
Tobias Peggs: We launched the Alpha version of the site in November last year, then we launched the Beta in May. But we have been building the product for just over two years.
There's some very complex search architecture in the background, which has to be put in place before you can put a site like this live.
When people have been using things like Google for 10 years, they expect things to happen, like page load times.
TR: Why do you think consumers are looking at real-time search now?
TP: People are realising there's a very real need for real-time search. If you look at the numbers and how they break down, regarding how people search the web, 20 per cent are navigational searches, those searching for Yahoo.com and Sony.com. 40 per cent are trying to find specific information: there's a recipe for cabbage soup that I need, and so on.
Traditional searches are very good at dealing with this 60 per cent of the market. But there's around 40 per cent of searches where the user behaviour is where they want completely up to date information on something like, say, the Iran election and Michael Jackson's death.
Consumers are beginning to realise there's new avenues to find this information and you don't have to just look on things like Wikipedia.
And with traditional search engines when you type in something it is typically these sorts of pages that come up first, and they fail to tell you what is actually going on right now.
TR: This is something that Twitter does…
TP: Twitter is helping with that enormously because it is such a buzzy, right now, place for content. Twitter is a Black Swan – it's just phenomenal.
TR: Did you know Twitter would explode the way it did?
TP: Being based in San Francisco, it's such a close community you couldn't not realise that it was going to be a success. Twitter has been part of my 24/7 experience, and you could always see what was going to happen.
But what this does is open people's possibilities that you can get real-time information about what is happening right now.
The issue with Twitter is that you can type something you want to know about it, and search results. You'll get pages and pages of information regarding your search but not all if it is going to be relevant.
Say it's people tweeting stuff about U2 – some of them have links to articles on the web, some is opinion, some is irrelevant.
So what we've done at oneRiot is harness that social energy on the web, people sharing stuff with other people. We look at Twitter trends, what's going on the Digg pages.
We also have our own panel of users, about three million strong who in real-time tell us what is happening. So we harness all that information.
So when you search oneRiot we show you the news article you have got to read that day, because that is what everyone else is reading, the video that is popular, the blog commenter who is making sense of things. It's all about fresh, socially relevant information.
TR: How do you rank your pages?
TP: We developed something called Pulse Rank. We very much believe in the need to rank things. You've found something you want to learn about, but there are thousands of things about it, so rather than provide a real-time stream of things that aren't relevant. We rank the results in real-time, based on pulse ranking which broadly reflects current social buzz around content.
So, if the video is being shared a lot on Twitter and our panel are watching the same video a lot, then this will be ranked higher.
So, it might not actually be the freshest thing, but it should be the most relevant.
TR: How do you get on the panel?
TP: It's more like an internet measuring service. So you join the panel and then as you surf the web, you can then pass that information back to us anonymously if you so wish, and we can use that information to find out what is hot and what is not.
The way that traditional search works is through indexing and ranking. So the way that Google indexes is that it links one piece of content to the other and this spiders around the web.
In terms of ranking, page rank is basically heavily based on citation, so the number of links to a certain piece of content. So, if you have written an article and lots of other people have linked to it, then that page increases in page-ranking and so on.
As a consequence, this tends to favour the well-established websites. So, if you were to Google Iran, the first hit would probably be Wikipedia and the second page maybe the CIA page on Iran, as these will be well-linked. it's a dependable system but doesn't work with real-time search as well.
So, we at oneRiot sat there and thought how we could integrate what's happening now into this system. In order to do this, we had to invent a new way to index the web and also to rank the content.
The indexing comes from what it happening on the social web right now, and the ranking is what's the most socially relevant content within this – what is the must-see video to watch.
TR: Does this mean that your system sometimes throws up some obscure blogs?
TP: That's one hundred per cent correct. It's like a meritocracy – if you write the blog that everyone is reading at the moment, then you will be on top. But we have to be careful that what is popular isn't spam, so we also look at who is sharing these links and if they are to be trusted.
Working with real-time means that things are constantly evolving, and so the search results on our sites are constantly evolving as well. That's why a lot of people who use our site use it multiple times in a day as they want to be kept as up to date as possible and stay on top of trends.
From a business aspect this is great for us, as because our users search many more times in a day with the same query, we have the potential to monetise that user many more times during the day.
TR: What do you think of Facebook buying FriendFeed?
TP: It's going to be interesting to see what happens with this. There is potential there, so if I want to watch a video that one of my friends posted a while back, then I'll be able to search and find it.
Until Facebook opens up the search though, it's going to be quite limited. If, say, I want to know a good coffee shop in London, I can search around my friends group to see if they have mentioned one before but the chances of this are unlikely.
There are definitely going to be some useful stuff but real-time search – tell me what my friends are thinking about this – you probably don't care, you would rather read the New York Times or something.
Having said that, with 220 million users, if they could turbo charge that content and open up all that content like Twitter has done, then that will become a really interesting core of information.
For more on the OneRiot search engine, www.oneriot.com. Or you can check out the company's new RiotFeeds – a micro-site powered by OneRiot that offers an easy way for tweeters to discover the hottest links in their Twitter stream for any topic- tech news, gadgets, sports, green living, food & wine. Check this out at http://feeds.oneriot.com.