Power up your website with the might of APIs

A tweet in time

Recently, Twitter has become a media darling and developer favourite, its simple concept providing basic building blocks ripe for repurposing. Pulling tweets into a blog maintains life signs, and Twitter is unparalleled as an early warning news source. "Twitter's also new to the layperson, so providing your latest tweets can show someone how Twitter can benefit them," says Miller.

"At a more advanced scale, enabling users to add tweets to an article that's also added to their Twitter feed is excellent advertising for your site and simple for the user, encouraging engagement." Twitter's relative simplicity enables radical reworking, too, such as accessibletwitter.com's alternative interface for users with specific access requirements, and 'visualisation', which Heilmann interprets as "using the power of APIs to make information more relevant by showing it in a different light".

By example, he mentions his script that displays your 'Twitter type' – a pie chart detailing whether you're a 'follower' or 'being followed' – which also uses the Google Chart API. Heilmann prefers the likes of Twitter to "massive Facebook-style catch-all APIs. I like seeing the web as a collection of data and helpers for me to make sense of it," he explains.

"A good API does one job and does it well, but developer programs like Facebook lock me in. When using them, I build for Facebook and not the web as a whole. This is when an API becomes a developer program or network. In a perfect set-up, you could use parts of a framework without having to subscribe fully to the network."

Heilmann says this is the case with Yahoo Pipes, which he uses to filter tweets: "Every tweet ending with § goes to my blog, ensuring I don't flood it with information nobody needs there." Adams also recommends Pipes, calling it "one of the most interesting web application interfaces and the most appropriate use of visual programming I've seen".

Although Pipes enables complex interaction between disparate data sources, Adams says he often uses it for relatively simple transformations, such as filtering items from RSS feeds, or accessing RSS via Ajax by transforming any RSS feed into a JSON-P object. "As with many useful APIs," he says, "Pipes fills a need you never knew you had, acting as the blender of the web."

The dark side of APIs

It's not all sunshine in the world of APIs, though. Newcomers must beware of many things before taking the plunge, not least that over-reliance on APIs can spell disaster if a service fails; and vendors can vanish or amend APIs on a whim, giving you no option but to rewrite custom code.

"Another issue is getting access to APIs," explains Heilmann, who recalls signing up for the TFL developer program to create a mashup of UK traffic cameras, only to discover a complex sign-up process asking for the range of IPs the hack would run on and requiring his email be part of the API URL: "That's needlessly hard for me to do and my email shouldn't have to be readable only to access data."

Worse is the proprietary nature of APIs and the variance in how they work. "Each and every one expects different parameters, returns data in different formats and has different terms and conditions," complains Heilmann. The first two issues can be overcome using YQL Open Data Tables, he says, but terms remain a thorny issue.

"My official Guardian hack violated their terms, due to me adding paragraph tags to the content to make it more readable – this meant I'd modified the content." You're also limited by what an API enables, unless you're willing to stretch what's officially allowed via hacks or workarounds.

Adams says most of his frustrations with APIs are summed up by one example: "I wanted to get a user's second level Flickr contacts. This is possible, but you must make a separate API call for each contact, and this quickly multiplies the calls you have to make, ultimately making it unfeasible to do such operations unless you're prepared to pre-crawl the data."

Adams notes that if you had access to the Flickr database, this operation would be trivial, but access via the API is subject to network latency restrictions, API rate limiting and various callbacks, and so not everything can be done with APIs.

Dixon suggests this is just as well, as he thinks APIs are overused: "Too many developers don't consider what a site really needs. This reflects badly on the developer and results in sites becoming interchangeable." He cites countless mashup sites that "do little more than aggregate information from other websites, and give a bad name to APIs".

Looking forward

Drawbacks aside, it's clear that APIs are now a cornerstone of web development, and even with the most popular of APIs, Dixon concedes there's hope: "When businesses think of innovative uses for the Twitter API to benefit them, that's when it will become successful long-term – not with these 10-a-day sites that tell me how often someone mentions my name on a tweet."

Undoubtedly, this consideration (once the 'new toys' mentality wears thin) mixed with existing and new technology should confirm rapid escalation of API use. In technology terms, it's OAuth that's currently getting many developers excited. Çelik describes it as "an essential authorisation building block for the continued growth and development of the web," adding that it "enables different websites to perform various functions for a user, while allowing that user to share private data across those websites with specific authorisation".

Miller thinks along similar lines, suggesting OAuth has the potential to eradicate the main barrier in encouraging online interaction: the hassle of signing up. Some developers remain negative, citing OAuth's awkwardness and difficulty, along with logistical concerns.

Inman, for example, comments: "There's redundant programming overhead, since you must implement traditional login for those without an OAuth provider; and as a developer and user, OAuth introduces a third-party whose reliability and longevity cannot be ascertained." But OAuth does hint at a near future where developers are relieved of figuring out how to handle sensitive information, yet can create increasingly elaborate mashups by pulling data from a mixture of different social networking applications.

Standardisation elsewhere should be of further help, enabling services to increasingly be 'glued together', while the likes of YQL Open Data Tables will become more popular, according to Heilmann, owing to them "providing an easy way to have an API without having to create your own back-end or document your output formats".

Organic evolution, with APIs increasingly worming their way into basic workflow, should also cement their purpose and popularity. Charlton notes that "APIs increasingly aid in more than adding functionality – they're used to provide the very infrastructure sites run on". He considers Amazon Web Services a great example. "You can store all your website assets (S3) and data (SimpleDB), and through an API add more computing power (Amazon EC2) to your application," he says.

Ultimately, what we're seeing at the moment is just the beginning. As Amy Hoy concludes: "I don't think we've gone near the limit of what can be done with Twitter or any other API. These things look simple, but they run deep."