Inside Apple’s quest to add a heart-rate sensor and full workout tracking into AirPods Pro 3

There’s no mistaking the fact that AirPods Pro are widely popular earbuds, and since their debut in 2022, we’ve seen Apple steadily add new features. Some require an easy, free software update, and others – like improved noise cancellation, longer battery life, or a case that is much harder to lose – require getting the next generation.

For AirPods Pro 3, Apple’s mostly kept the now-iconic design aside from a redesigned ear-tip that uses both silicone and foam and a look that takes its cues from AirPods 4, stretched the case a bit, and oh, the engineering team managed to add in another sensor. That addition was in the form of a heart-rate sensor, which allows you to track 50 workouts with just your AirPods Pro 3 and an iPhone, or let them work alongside an Apple Watch to pick the most accurate and stable reading.

Now, I’ve already detailed my early experience with AirPods Pro 3 – spoiler, the improved ANC and better sound won me over – but here I’m diving deep into arguably the newest innovation, the heart-rate sensor. TechRadar had the opportunity to speak with Apple’s Steve Waydo, Director of Health Sensing, and Ron Huang, VP of Sensing and Connectivity, to delve deeper into the new sensor: its development, how it works, and ultimately what its arrival means for AirPods Pro.

The heart-rate sensor in the AirPods Pro 3 is not Apple’s first foray into this sensor type. Apple initially offered such a sensor when the Apple Watch first launched in 2015, so as Waydo explained, “it was really cool here to get to bring everything we learned over more than a decade of that work to this very different form factor.”

Shrinking a decade of work

Waydo's team at Apple has been working on heart-rate sensing algorithms since the original Apple Watch, and that 10-year journey proved helpful in the development of the new sensor – but it’s not one for one. “The heart rate sensor in the AirPods Pro uses invisible infrared light. We pulse it up to 256 times per second, and we take that data and we fuse it together with what we're getting from the onboard accelerometers to measure the blood flow in your ears,” explained Waydo, who noted that’s where the final measurement for heart rate comes from.

This differs from the Apple Watch, which uses green LEDs. Here, the sensors are invisible, and there are either two sources or a single, as AirPods Pro 3 can provide a heart-rate reading with just one bud in or with both in. With the latter, the algorithms work in real time to pick the best, most accurate source.

On those algorithms, the base for how AirPods track heart rate comes from the Apple Watch, but Waydo explained that given these are much smaller, the team “had to shrink those algorithms down to fit within the processing and memory constraints, so that we could deliver the same kind of speed, efficiency, and battery life that our users really love.”

To get there, the team had to ensure the new sensor would work for everyone. As Waydo explained, “there's a huge amount of variation in ear geometry that we had to tackle, both through fit and through our studies to understand if we could get great signals across everybody who's going to use these AirPods.”

That meant testing across thousands of participants, all with unique ear shapes and different skin tones across a range of environments and workouts, to ensure good contact with the sensor. At the heart of it is machine learning: “This is a neural network algorithm that uses machine learning across many thousands of people who have been through our user studies,” which dates back to ones conducted for the Apple Watch.

Apple did have to adapt these for collecting the data from an ear versus a wrist, so reading new signals, as well as having to “miniaturize these algorithms to run in that more constrained environment,” said Waydo.

Ultimately, with the consumer release now available, the algorithm can adjust for these different conditions. All that study and work also helped identify the correct spot on the individual AirPods Pro 3 for where the sensor should reside. The result is that the sensor is placed in a location that, across a wide range of people, provides good contact and signals to effectively measure heart rate.

It’s also something of a conundrum to figure out where the heart-rate sensor physically fits inside such a small product – Waydo’s team had to determine just how small it could be and the proper orientation, which resulted in them going through “many iterations of trying and modeling and measuring different combinations of positions to find the overall best combination for the product as a whole.”

Beyond the heart-rate sensor

That’s a lot of work just for the heart-rate sensor itself, but Ron Huang also delved into the algorithms, how they work with other hardware, and ultimately how they move AirPods into fitness tracker territory.

Considering it’s Apple, the team did start with an ambitious goal: “For AirPods, our goal is to bring all 50 workouts to our users all at once, all at launch.”

That’s a stark contrast to the Apple Watch, which has steadily gained new workouts since its launch. To achieve this new goal, Apple built what Huang described as a “brand new motion foundation model …, that really captures the human motion holistically.”

Over 50 million hours of workout data Apple collected through its Heart and Movement Study was the starting point, and was then adapted for AirPods. That requires the heart-rate sensor to work alongside accelerometers and gyroscopes, tracking a broad range of workouts. Those sensors in the AirPods are combined with “additional algorithms to track steps, distance, and calories,” said Huang, noting that GPS and barometer data from the iPhone completes the picture.

Additionally, Huang shared that workouts like functional strength training and pilates required additional fine-tuning to accurately track calories. Further, to validate all of this, Apple relied on its biomechanics labs.

“We have treadmills that are highly instrumented and highly calibrated, so we know the perfect distance that you are running,” explained Huang. The lab also uses insoles with pressure sensors to calculate exactly when a foot is lifted or placed down and FPS cameras to “track and capture the full gait motion” of the participants. Those tools enabled millisecond-precise tracking of heel strikes, toe lifts, cadence, and stride, which were used to train a neural net model for steps and distance.

To ensure accuracy for calorie tracking, Apple opted to verify it against medical-grade equipment. “The gold standard for measuring calorie expenditures by putting a mask on you and measuring the oxygen output, so that's what we did through these studies to make sure that the AirPods and phone algorithms are accurate here,” explained Huang.

That’s a significant amount of effort, time, and work invested in ensuring the level of accuracy for workout tracking on AirPods Pro 3. Huang noted that while a lot of learnings were gained from the watch, it also involved a significant amount of AI and ML work to build these algorithms, which resulted from collaboration across a wide set of teams at Apple.

From AirPod to iPhone

I was still left curious about a few things with AirPods Pro 3 for workouts – mainly how the data is packaged for processing. Huang explained that prior to this additional data set, Apple was already squeezing a lot between the Bluetooth link of AirPods and the iPhone. It started as motion data for Spatial Audio, and now it’s heart rate alongside other data; Apple is essentially optimizing the connection here.

“The motion data from the accelerometer and gyroscope are forwarded directly from AirPods to iPhone at 100 samples or 50 hertz samples. Heart rate is computed from the raw signals to BPM values that we send over to the iPhone as well. The actual caloric algorithms are all running on the phone side,” explained Huang.

Even so, if you’re wearing an Apple Watch at the same time, there is additional processing to look at the heart rate from that wearable, as well as the left and right AirPods Pro 3. Prior to the AirPods Pro 3, if there was ever low confidence in a heart-rate reading from the Apple Watch, that set of data wouldn’t be visible to the user; a gap would be present. So AirPods Pro 3 solves for that a bit.

Waydo explained that these algorithms run independently in up to three areas, and the algorithm “looks across all of those sources and decides which one is most confident, which one has the best signals to produce the best possible answer.”

Considering it could be getting a reading from both the wrist and the ear, it also accounts for other factors that could impact the reading. Summing it up, he said, “In any given scenario, one or the other is pretty likely to have a very good signal.”

I’ll close with something Huang said when I asked about the evolution of AirPods Pro 3. They now pack in heart rate monitoring, workout tracking, and a hearing health suite, on top of everything you’d expect from premium earbuds. So what are they, really: a super pair of earbuds, a fitness tracker that plays music, the reverse, or something else altogether?

“Each year, we're able to keep up on adding just really nice features for our customers. So I think AirPods is a lot of things for a lot of people, and different things for each person.”

Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds. Make sure to click the Follow button!

And of course, you can also follow TechRadar on TikTok for news, reviews, unboxings in video form, and get regular updates from us on WhatsApp too.

You might also like

• ‘Those phones are being treated just like any other camera’ – Inside Apple’s first Friday Night Baseball broadcast shot on iPhone 17 Pro
• Disney Imagineers built a new animatronic parrot for the Beak and Barrel lounge – and it steals the show
• Apple took me into 'one of the quietest rooms anywhere in the world' and now I understand its obsession with AirPods Pro sound quality