Meta is saving millions of dollars thanks to this clever memory hack

Inside Facebook data center
(Image credit: Facebook)

Engineers at Meta, Facebook’s parent company, have revealed how they have been able to offer free memory using a software solution called Transparent Memory Offloading (TMO).

It is now part of the Linux kernel and, in a nutshell, automatically offloads data to other storage tiers (e.g. Samsung’s CX memory expander) that are less costly and more power efficient than memory.

The savings are significant; TMO has been running on millions of Facebook servers for more than a year, saving up to almost a third of memory per server. While that is likely to be insignificant across dozens or even hundreds of servers, Facebook’s immense scale presents a unique challenge. 

Analysis: Facebook's gargantuan appetite for RAM

The world’s largest social network has nearly three billion monthly active users and millions of servers spread around 21 locations worldwide. Should each server carry  128GB of RAM on average, that would amount to 256 million GB (or 256PB) of RAM which, at an average cost of $4 per GB (DDR4 ECC RAM), is about $1 billion worth of memory. That’s on the assumption that Facebook has at least two million servers (Facebook’s blog quoted “millions of servers” as early as July 2018), with the real number likely to be far higher.

Numbers presented by the team that worked on TMO showed that the cost of memory accounts for a third of Meta’s server bill of materials, with compressed RAM and SSD accounting for less than 11%. More worryingly, the cost burden of RAM (as a percentage of the total infrastructure) has more than doubled since Facebook launched its first generation of servers (it's currently on the fourth).

Adopting TMO does come with some drawbacks; most notably, a degradation in performance. But the gains in terms of power and memory savings, far, far outweigh the disadvantages and future iterations combined with hardware improvements (e.g. faster SSD or CXL drives) will offer further mitigation. 

TOPICS
Desire Athow
Managing Editor, TechRadar Pro

Désiré has been musing and writing about technology during a career spanning four decades. He dabbled in website builders and web hosting when DHTML and frames were in vogue and started narrating about the impact of technology on society just before the start of the Y2K hysteria at the turn of the last millennium.

Read more
A person standing in front of a rack of servers inside a data center
Changing a few lines of code in Linux could apparently save hyperscalers billions, research claims, but I am not convinced
Meta QLC Server
Facebook engineers say bigger hard disk drives is making one critical metric far, far worse
Quantification of orbital torques
We're getting achingly closer to SOT-MRAM; the memory that could one day replace DRAM and NAND, but I can't see it happening soon
Sandisk 3D Matrix Memory
Sandisk's revolutionary new memory promises DRAM-like performance, 4X capacity at half the price
A person standing in front of a rack of servers inside a data center
Microsoft joins scientists in finding a way to reuse decommissioned servers
d-Matrix Corsair card
Tech startup proposes a novel way to tackle massive LLMs using the fastest memory available to mankind
Latest in Pro
Lock on Laptop Screen
Data breach at Pennsylvania education union potentially exposes 500,000 victims
Spyware
Stalkerware data breach potentially hits over 2 million users, including thousands of Apple devices
An American flag flying outside the US Capitol building against a blue sky
Five Eyes "cannot replace US intel in Ukraine", claims former US Cyber Command Chief
An AI face in profile against a digital background.
Getting your data ready as the AI race heats up
Pirate skull cyber attack digital technology flag cyber on on computer CPU in background. Darknet and cybercrime banner cyberattack and espionage concept illustration.
Criminals are using a virtual hard disk image file to host and distribute dangerous malware
Oracle
Oracle unveils multi-billion dollar investment in UK cloud and AI
Latest in News
Quordle on a smartphone held in a hand
Quordle hints and answers for Friday, March 21 (game #1152)
NYT Strands homescreen on a mobile phone screen, on a light blue background
NYT Strands hints and answers for Friday, March 21 (game #383)
NYT Connections homescreen on a phone, on a purple background
NYT Connections hints and answers for Friday, March 21 (game #649)
The ASSC Assassin's Creed collection.
The Assassin's Creed x Anti Social Social Club drop includes gaming merch that I wouldn't be embarrassed to wear
Lock on Laptop Screen
Data breach at Pennsylvania education union potentially exposes 500,000 victims
Spyware
Stalkerware data breach potentially hits over 2 million users, including thousands of Apple devices