Facebook and open source: 'we've come a long way'

David Recordon, Facebook's head of open source

Facebook and open source

With over 500 million users, Facebook is by far the world's biggest social networking site - and has also been making important contributions to the open source community.

David Recordon, the site's head of open source, talks us through the company's approach to free software.

LXF: What's your background in open source?

David Recordon: I really got started working with open source when I was a teenager. I was using YaBB SE (PHP forum software), and started helping others within the community.

At the time, I knew C++ and a bit of Perl, but really hadn't done much web programming. PHP was easy to pick up and I loved the immediacy of being able to just hit save and then refresh my browser. Over the next few years, I got more deeply involved in the project, helped launch the rewrite as Simple Machines Forum, and built a forum-hosting business with my friend Joshua Dickerson.

I guess this was really my first experience of being deeply involved in an open source project.

LXF: Did you continue working in the web business?

DR: A few years later, I interned at LiveJournal, really learned Perl, and was told to never admit that I used Nano (after switching to Emacs). Working at LiveJournal was also my first experience of using Linux on the desktop daily.

LXF: You're well known in the OpenID world. How did you get involved in the project?

DR: A number of people took advantage of the fact that LiveJournal's core was open source and ran clones. According to Wikipedia, there have been 30 different sites, with DeadJournal being the first in 2001. While spam was never a major problem within LiveJournal.com, commenting across blogging sites was still riddled with it, because there was no shared notion of identity.

OpenID was originally created at Six Apart to help tackle cross-site commenting in a decentralised manner. It's obviously evolved quite drastically since 2005, and I really got deeply involved in OpenID 2.0 while I was at VeriSign.

It's been implemented by just about every major web company, but we still have a long way to go. I see the next version of OpenID being built on top of OAuth 2.0. This will allow it to be a fairly small piece of technology that should work for applications outside of the browser.

LXF: How did you make the move to Facebook?

DR: I joined Facebook in 2009 to work on open source and standards. My team is focused on making it easy for anyone in the company to do a really great job of using, contributing to and releasing open source projects.

At times this means embedding ourselves within other engineering teams, such as with HipHop for PHP, which we released in early 2010. While HipHop had proven itself within Facebook, there was a lot of work still to do to make it a useful piece of infrastructure for others.

LXF: What exactly does HipHop do?

DR: HipHop really embodies how we create at Facebook. It started as a hackathon project by Haiping Zhao, who was later joined by Iain Proctor and Minghui Yang. Haiping noticed a number of similarities between the syntax of PHP and C++, and wondered if you could programmatically rewrite one into another.

Two-and-half years and a few other engineers later, HipHop was serving the vast majority of Facebook's production traffic. It takes our PHP source code, transforms it into C++, and compiles it into a self-contained binary that we deploy on production web servers.

This is a typical pattern of how projects get started at Facebook. One or two people just decide to try something. We refer to this as our "hacker culture".

LXF: Is your work on the OpenID and OAuth standards used at Facebook?

DR: Our platform engineering team did a lot of work on OAuth 2.0 this past year. OAuth was created to standardise an API design pattern where people could grant websites access to act on their behalf without having to share their password. While OAuth 1.0 was used in just about every new API over the past two years, it was too complex for many developers. We helped create OAuth 2.0 to fix that.

OAuth 2.0 relies on SSL to protect an access token when making API requests versus HMAC signatures, which were used in 1.0. This removes a great deal of complexity, because developers interacting with your API no longer needed to normalise, sort, and then sign all of their HTTP request parameters.

We were the first to ship OAuth 2.0 as a part of the Graph API announced at f8 in April after working within the IETF community to write a good deal of it. Also at f8, we introduced the Open Graph Protocol (http://ogp.me), which uses a very simple subset of the RDFa framework to represent any web page as a part of a social graph.

LXF: People think of Facebook as a closed-source site. How else do you contribute to open source?

DR: Facebook engineers actively contribute within the Apache Hadoop ecosystem and to MySQL and PHP, and have created a number of features that allow memcached to scale on modern hardware.

But we don't just contribute to other projects or release developer tools, we open source entire pieces of production infrastructure. HipHop, FlashCache, Apache Hive and Cassandra, Thrift, Scribe, and others were all created at Facebook. I don't think that there's another web company of our size that's done the same.

LXF: On the flip side, what's difficult about working on open source within a company?

DR: It's easy for companies to fall into believing the myth that open source doesn't take additional time and effort. It really does take time if you're going to do a good job. And it's important to properly set expectations around projects in terms of the spectrum between just sharing your source code under an open source licence and fully sharing control over the project itself.

I almost think the decision here is less important in comparison to companies appropriately setting expectations.

LXF: What's your view on Facebook clone Diaspora?

DR: I have a lot of respect for those guys. They're obviously passionate about what they're working on and are actually building a product. I think there are a lot of challenges in what they're trying to create.

An open source social network is about far more than status updates and sending messages between sites; it's also about having a global sense of identity and bringing both your friends and your content with you around the web, while keeping you in control over who can see what you've shared. It's about building a platform.

LXF: Do you feel that Facebook is currently embracing the open web?

DR: Yes, I think we've come a long way in the past year. Whether it's OAuth 2.0, HTML 5, or the Open Graph Protocol, we've used standards where they exist and worked with the community to create them in some of the areas they don't.

We'll often get criticised for not implementing a given technology, but the best standards are created following working implementations and not the other way around. As I wrote over the summer in reference to emerging standards: "Don't be afraid to rip them apart as needed if you'll end up with a better product, a better technology, and ultimately a better standard. We did this recently with OAuth 2.0 and the internet is better for it"

Article continues below