How Metalink gives you faster downloads

Want to make your downloads easier and faster? Then it's time for a quick lesson in Metalink...

You're wrong I tell you! BitTorrent is the One True Way of downloading files, and I won't hear any different!

I see. In that case, why not check out TechRadar's article on speeding up BitTorrent downloads. But if you'd like to hear why Metalink is designed to sit alongside BitTorrent rather than replace it, hang around and I'll tell you.

Still here? Good. Now let me tell you a common problem with BitTorrent: people hate it for casual downloads. Sure, it's great for snagging the latest Fedora DVD ISOs, but if all you want to do is download OpenOffice.org or Firefox, BitTorrent seems like a lot of hassle.

I'd agree with that much – if I ever see a direct download link next to a torrent, I have to admit to using the direct download. You won't tell anyone, will you?

No need to feel guilty; it's a very common thing to do. But of course the problem with direct downloads – the reason we all tried to stop using them in the first place – is that they put a lot of strain on server bandwidth, which means you end up having to choose from a list of mirrors and hoping you find one that isn't too full. Sometimes your best bet might even be to use FTP!

And Metalink helps me how?

Metalink is a very simple XML-based file format that contains multiple sources for a file, letting you choose how you want to get that file. For example, if you want to download Fedora, PCLinuxOS, SUSE, Ubuntu or a dozen other Linux distros, you'll find they all use Metalink – and you just need to snag the metalink for the file you want in order to have it do the smart thing.

OK… so what's the smart thing, and how does Metalink choose it?

If you're in the US, the smart thing is to download from the recommended US HTTP server. If that's full, the smart thing might be to go straight to BitTorrent as a backup (if one has been configured for the Metalink), but equally you might have configured your download client to try other HTTP servers first.

So Metalink automatically knows which servers are best?

It makes an educated guess, based on the location of the server and the preference for that download option. Both of these are set in the Metalink file itself, allowing sysadmins to encourage users of the system to download from a local server and download from the fastest server too.

That all sounds very nice, but won't I lose some of the advantages of BitTorrent if I switch to Metalink?

Well, clearly you won't lose any of BitTorrent's features if Metalink falls back to using BitTorrent for a download, and Metalink borrows some of BitTorrent's smartest features even when it's not using BitTorrent. The example you just gave is known as checksumming, because BitTorrent can calculate a simple number that is almost unique to a file, even if that file happens to be a giant 4GB DVD ISO image. Metalink also supports checksumming, which means you get the same error resilience even when you're not downloading using BitTorrent.

OK, but BitTorrent already has it. Does Metalink have any real advantages over BitTorrent when downloading large files?

Yes, yes, yes, I was just getting to them! How about PGP key signing, allowing you to be sure that the file you're downloading definitely comes from the person you think it does?

Meh.

OK, how about segmented downloading? That's where Metalink will download the file from multiple sources at the same time.

That sounds completely pointless – I only need the file once!

Well, yes, obviously. But Metalink only downloads parts from each source, meaning that you can saturate your bandwidth without slowing down one server – Metalink spreads the load across several options. So if one file has a dozen mirrors, Metalink makes it easy to use them all at the same time to get the file super fast.

So BitTorrent really has nothing to worry about from Metalink, right?

Right. In essence, Metalink takes a lot of the lessons learned from doing BitTorrent and applies them to other downloading systems as well, then brings them all together in one place. Metalink is about making downloads easier.

But I already have a BitTorrent client in Transmission, and there's no shortage of alternatives out there – Monsoon, rTorrent, and so on. What's your point?

Just that it doesn't get much easier than BitTorrent because most distros already have BitTorrent software as standard!