The proof is in the pudding, as they say. At Yahoo, the pudding is quite murky at present, what with Jerry Yang out as CEO, a disastrous 2008 in terms of stock price and failing to be taken over by Microsoft. Yet, just in the past few weeks, as Carol Bartz took over as CEO, it's as though the dark cloud has lifted.
Truth be told, Yahoo has just as many Stanford grads and doctorate researchers on staff as Google, and has always tried to push the envelope in terms of web innovation. And it must: future technologies on the web - such as location-awareness, social networking, cluster computing and advanced web search algorithms - do not just spring to life, they must be born and bred in a research lab.
In a recent visit to the Yahoo facilities, we met with the company's top researchers and innovators, including the VP of engineering and a very smart Oxford alumnus working on mapping technologies. It was an illuminating experience, because we realized that Yahoo is not just another me-too search giant or a company that acquires start-ups like Flickr.com. Here's the most exciting projects we found...
Yahoo seeks to make more of its web properties as open as possible. Today, disparate web entities within Yahoo - such as Answers and Search - do not share data well. And Yahoo doesn't do it well with other sites, either. Y!OS intends to rectify that problem. And we're already seeing the earliest signs of this kind of interoperability with sites such as Twitterfeed.com that let you log in to the site with your Yahoo account. Neal Sample, the Vice President of Engineering, says there shouldn't be a multitude of logins while APIs should be more open (so developers can build features into Yahoo), standards like XML should be more portable, and standalone apps should be easier to create.
"Users should be able to bring other experiences into Yahoo," says Sample, who we met with at Yahoo HQ in Sunnyvale. "With Y!OS, developers could add to existing services, such as Yahoo Mail or take an existing product and expand it. We allow developers onto the network and give them the ability to augment services. By opening data and preferences, we want to platform-ize the Yahoo experience."
If this all sounds familiar to you, it's because operating systems went through the same phase early on – unifying platforms and data models, making tools more widely available, and encouraging application development. Will it lead to a Yahoo web OS someday? Sample would not make any specific predictions, but did agree that web standards such as OpenID are paving the way for a web OS.
Yahoo has a keen interest in cluster computing, which uses commodity hardware to handle the most difficult and complex computing tasks. The goal of the Hadoop project is to mask the complexity of the hardware but to allow parallel processing tasks to run quickly and efficiently.
"We enable people to explore problems they could not address in the past, and to innovate with data," says Ajay Anand, part of the Yahoo Grid Computing initiative. "Yahoo Search was one of the first projects that benefited from this scalable structure. The intent is to do this processing as fast as possible and as cheaply as possible. With thousands of machines, something will break almost every single day" which is why the Hadoop project tends to focus on a scalable infrastructure.
In September of last year, Hadoop was scaled up to 4,000 nodes, each powered by two quad-core Intel Xeon processors running at 2.5GHz, 4x1TB SATA disks and 8G RAM.
3. Marketplace design
We're hesitant to include a project related to web advertising. After all, most of us cringe at the subject of inter spatial banner ads and textual links that relate to our private emails. Yet, a new trend in web advertising is to apply heady economic principles; it's become more about the science than the marketing.
For example, eye-tracking is often used to determine how you read a website, and economists now know that ads must be arranged with an informative banner in one location and a separate 'action' button in another. We're so used to banners we rarely click them any more.
The Marketplace Design project intends to unify advertising. For example, if someone wants to advertise on Glam.com and Yahoo Shine at the same time, they have to buy and track two different ads.
Preston McAfee is a Princeton-educated economist and now a researcher at Yahoo who notes that the current ad model on the web is a bit dysfunctional. There are 400 different ad networks and none of them can exchange data, yet only four or five companies actually host ads and analyse the economics. Understanding the ad data is a monumental engineering challenge.
"You click on some ads and you are not sure whether it will take you where you want to go," says McAfee. "People are not sure if an ad will take them to an unknown site. More information on ads informs the user that where they click will be valid and worthwhile."
4. Zync Messaging
There was a startling moment during a presentation and hands-on demo of the Zync messaging service. The online chat looked normal enough, but at one point, we saw a link to watch a video at the same time as a friend. Today, the web is not good at concurrent engagement, or the ability to experience the same piece of technology at the same time as another person.
People remember data on the web differently. One person might remember a specific video and a link, another might only remember who they were chatting with when they watched a video. Zync – as a research project – shows how the people and the content can be linked more closely.
"If you send me a link to a video, I might or might not click on it," says Elizabeth Churchill, a Yahoo researcher who works on social systems. "But if we can watch the video together, I am much more likely to click on it – there is a social obligation. We have decreased the failure point."
Another interesting research project called MapChat enables users to chat on top of a map at Yahoo Maps. With loose ties to the geo-spatial research, MapChat links social networking characteristics to geographic data so that you can chat with someone based on their location or any point of interest on the map.
Elizabeth Churchill says that the social engagement of tools such as MapChat is much higher. And, there's an economic factor (always critical at companies such as Google and Yahoo) because it means the site is 'stickier' and will bring users back so they will see more ads and exchange information with other users.
6. Geospatial analytics
The web is a vast wasteland of disconnected information. You can post a blog entry, or an image, but the data is often disconnected from where you posted it and where you took the photo. Yahoo has a geo-spatial analytics project that is analysing how to scale geographic data.
Tyler Bell, an Oxford-educated expert on geo-informatics who heads the Yahoo Geo team, says geographic information on the web is not just about displaying data on a map. It's the who, what, where and when of web data.
"We focus on geo-relevance, not just displaying polygons on a map, but making sure all the info displayed at Yahoo and the internet is geographically relevant," says Bell. "We take the content and what it relates to and find where the user is, and match them appropriately. As people use the web – and this is still 12-18 months out – we will be able to tell where the information comes from."
7. Open Mail
Like the Y!OS project, Yahoo Open Mail (currently in beta) lets you access sites like Flickr.com from within your mail account – such as, creating a new message that allows you to see Flickr images from within the body of the email and select the pictures you want to send. From within the Mail client, you can also browse images and delete the one you don't want any more.
Open Mail supports other Yahoo properties such as Yahoo Greetings, but the plan is to expand it even further, perhaps unifying other email messages from sites like Facebook and MySpace into one inbox.
You might also like Is Apple planning a DVR and web-enabled TV set?
Sign up for the free weekly TechRadar newsletter
Get tech news delivered straight to your inbox. Register for the free TechRadar newsletter and stay on top of the week's biggest stories and product releases. Sign up at http://www.techradar.com/register