Having kickstarted the Data Liberation Front movement inside Google, and as an advisor on open source matters, Brian Fitzpatrick knows the inner workings of the search giant very well indeed.

We caught up with him at OSCON in Portland, Oregon.

Linux Format: How did the Data Liberation Front come about inside Google?

Brian Fitzpatrick: It came from a number of things. I guess it first started when I graduated from college, and I couldn't take my email with me. And when I got to Google, I did open source stuff for a year and a half, or two years, and we were asking: "What else can we do in the Chicago office that's philosophically equivalent to open source, but not necessarily open source?"

I talked to a lot of people, and we were told to always focus on the user. I thought, 'I'm not going to be the guy to make Google the next two billion dollars – but what can I do that's going to make a big difference?'.

So I saw that we're getting to a point where big companies have a lot of data – where people are storing data in other companies. We don't need another kind of lock-in in the world, right?

The way I saw it was, Google never locked users in for search. Why do you use search? Is it because you have a two-year contract? Did you buy a piece of hardware to use it? No! If you don't want to use it, you just use another search engine. Who gives a damn?

The way we keep you as users is to make it better. Rapid innovation, rapid iteration. So we thought, 'If we make it even easier for people to leave our products, we're going to be forced to iterate even more quickly, and make our products better'. Everybody benefits from that, right?

Users benefit from it, and we benefit because we're competing really fairly. I mean, as an engineer, I'd much rather build a better product than build bigger walls around a product.

LXF: Kind of unusual...

BF: Well it is unusual, but the thing to remember is that, today, we're in an unusual world. Fifteen years ago, if I had told you, "In 2011 there's going to be a global distribution centre that costs almost nothing to send things on," what would you say? You'd be like, "Oh come on! Is it teleportation?"

But here we are in 2011 with the internet. It's a game-changer – it has changed the game in so many ways. And this is another one. It's really easy for people to switch software, and it's really easy for people to try new software.

And that's great – people who, in the past, have been very afraid of computers embrace them. They'll try new things really quickly.

LXF: Was the Data Liberation Front scratching an itch within Google – like engineers wanting to move data around in their own projects?

BF: Sure, as an engineer I don't want to waste my time schlepping bits from one place to another. If it's easier for you to take it out, it's easier for you to get it in. We want to go both ways on that.

I think a lot of people misunderstand why we do it. I think they understand why we do open source – they say, "Oh, Google wants everyone to think they're nice and cuddly." But the fact of the matter is, it's good for our business. It keeps us competitive. If we start building bigger walls, and leave our products to lie fallow, what's going to happen? Some start-up who really wants our customers is going to come and build an amazing product. Any other competitor can do it, and they take our users away.

LXF: It must be a hard thing to pitch to the bean counters, and the people controlling all the money at the top…

BF: I was a little reticent at first, thinking, 'Am I going to get fired?'.

LXF: So it was you that actually pitched the idea then?

BF: Sure. The first time I went and talked to Eric Schmidt about this, he said: "Why are you in here talking to me about this? Why aren't you out there doing it?" So off I went, and when everyone found out about it, they found it fits in with our mission and our philosophy.

I want to live in a world, and in the internet, in the future, where things are open and it's easier to move from one place to another. We're still very early in the life of the internet – a lot of the ways we deal with data are very specific to implementations. If you were to rent an apartment, and they said you can't take anything out when you leave, would you stay there?

LXF: If it was a really good apartment!

BF: Really? You'd leave all your family photos, wedding pictures? Would you check yourself into prison, automatically? People do this all the time with their data! That's the way I see it.

LXF: It must've been a pretty big technical challenge, if you were going from a standing start. Like with Gmail…

BF: Gmail is pretty big, but say that something is a "technical challenge" at Google and it's a little dicey, right? Because we're going to index the entire web, right? And we're going to serve up ads faster than you can sneeze. That's a technical challenge!

I would say there have been a lot of challenges in getting going, but those are about integration and doing things securely. Finding ways to build on APIs if they exist, or building new APIs if they don't exist.

LXF: Does it affect the way that new products at Google are designed, to actually incorporate the Data Liberation aspect?

BF: People are thinking of it sooner and sooner. People used to think about it after launching a product, and now people are thinking about it before launch. And teams have been reaching out to us to say, "Hey, how do we do this right?"

LXF: Do you see that becoming an official policy one day at Google? Like, if you make a new product there must be a way to get the data out?

BF: Well, I hesitate to make something an official mandate or policy, because when you come up with a policy there are always situations where an exception is worthwhile. My belief is to make it more part of the culture, so that it's something that people are voluntarily doing. They're thinking, 'This is something I want my product to have'.

So I'm not big into setting firm policy unless I feel there's a real danger to something or someone. For some products, depending on how fast they're iterating, it's easier to get something at launch.

LXF: At the moment, on the Data Liberation Website you get a list of products and methods for getting them out. Is the idea to move them all into Google Takeout, so eventually you have a big tarball that you download?

BF: I'd like to see everything in Google Takeout – we'll see if we can get there; that's a lot of work. But it's your data, and you should have control over it.

My thought beyond that is, people tend to mis-trust big companies – Google gets a hard time because of the whole 'Don't be evil' thing, but I'm glad of that. I'm glad that people hold us to a higher bar. People don't think of other large companies and say, "Oh my god, they treated me poorly!" No, they say, "They treated me poorly, and that's what I expected."

LXF: Our lives are in Google...

BF: Right. So if you say, "I don't trust Google," or "I'm not happy with the direction they're going," or whatever, you can take your data – your trust – and go somewhere else with it.

LXF: How popular has it been? Do lots of people use the service?

BF: Not a lot of people – we've been liberating data for years, and it's sort of like an emergency stairwell on a building. In the case of a fire drill, you take the stairwell out, but every day you use the elevator.

So it's not something that people use frequently, but the response we've got from users when they've heard about it is astounding. People have been really excited about it. They see it as a commitment – putting your money where your mouth is, so to speak.

--------------------------------------------------------------------------------------------------

First published in Linux Format Issue 154

Liked this? Then check outIs the future of search social?

Sign up for TechRadar's free Week in Tech newsletter
Get the top stories of the week, plus the most popular reviews delivered straight to your inbox. Sign up at http://www.techradar.com/register

Follow TechRadar on Twitter * Find us on Facebook * Add us on Google+