How the Huawei P20 Pro triple camera works

null

For years the chase for megapixels in phones has been forgotten. Huawei has just brought it back, though.

The Huawei P20 Pro has 92 megapixels' worth of camera sensor in its 7.7mm thick frame, including a truly unusual 40-megapixel primary camera on the back.

This isn't the first time we've seen a 40MP phone (who remembers the 808 Pureview and Lumia 1020?). But following several years of the best phones using 12-megapixel Sony sensors, you have to wonder what Huawei is up to.

How the camera setup works

The Huawei P20 Pro has three cameras on its back with Leica lenses: a 40MP one, a second 20MP camera and a third 8MP sensor.

Its 40MP camera serves several functions. First, it gives Huawei something great to show off. A 40-MP sensor sounds far more advanced than the iPhone X's 12MP one.

It also lets the phone take exceptionally detailed photos when there's enough light to avoid using high sensitivity settings, which introduce noise and reduce detail.

The high-resolution sensor is also used alongside the third of the Huawei P20 Pro's three rear cameras to take low-loss 5x zoom images.

This third camera has a 3x zoom lens in front of an 8MP sensor. When you zoom past this native 3x zoom to 5x, the Huawei P20 Pro reverts to a "hybrid" zoom strategy. 

The visual feeds of both the 8MP zoom camera and a crop of the 40MP wide angle camera are combined, giving the behind-the-scenes image engine more data with which to work.

The problem with high-resolution sensors

The issue with using very high pixel density camera sensors is that to fit in so many photosites - the individual light-sensitive elements that make up a sensor - each sensor pixel has to be very small.

In the Huawei P20 Pro, each sensor pixel measures one micron, or a thousandth of a millimetre. The standard P20 has 1.55 micron sensor pixels. Samsung's Galaxy S9 has 1.4 micron sensor pixels.

The smaller these pixels are, the less light they are able to "record" or collect in each, say, hundredth of a second of exposure.

It's doubly concerning when the Nokia 808 PureView and Nokia Lumia 1020, 40MP phones from years past, both had larger sensors. 

The 808 had a 1/1.2 sensor, for 1.4 micron sensor pixels while the Lumia 1020 had a 1/1.5 sensor and 1.12 micron sensor pixels.

However, mobile phone camera technology has developed since then. It's why the Huawei P20 Pro has a secondary 20MP black and white sensor.

This sensor has roughly the same view of a scene as the main one, and its feed is used in combination with it to record normal images. So why is this useful?

We have to look a little deeper at how a camera works to find out.

A lot of work has gone into squeezing three rear cameras in the P20 Pro

A lot of work has gone into squeezing three rear cameras in the P20 Pro

When you fire-off a shot, light passes through the camera lens onto the sensor. But before it reaches the photo sites, or sensor pixels, it hits a colour filter. 

This separates light out into the three primary colours, red, green and blue. Each photosite is responsible for recording one of these tones' colour information.

In a black and white sensor, colour doesn't enter the equation. The sensor pixels only have to worry about luminosity. And there's no colour filter, which discards light as it filters it into different colours. It makes retaining clarity and recovering dynamic range as the picture is constructed easier.

This should make more of the Huawei P20 Pro's megapixel count useful. Higher resolution doesn't always mean more detail out in the real world, but this will help.

Huawei knows this too. While the camera may have a 40MP resolution, you'll want to use the built-in 10MP mode a lot of the time. Not only will this result in smaller files, it will improve noise levels and lower-light performance.

This mode involves "pixel binning", where the information from sensor pixels is combined in a 2x2 pattern to reduce anomalous information.

Super Night: When in doubt, cheat

Some of the most impressive Huawei P20 Pro night shots you'll see online won't simply be a result of the combo of multi-sensor shooting and pixel binning.

The phone also has a Super Night mode. This is an "AI-fuelled" alternative to long exposure shooting. Over the space of a couple of seconds, the cameras take numerous shots at different exposure levels and combine them to produce a shot that looks like it could have been taken using an ultra-long exposure.

AI Image Stabilization

 AIS, or AI Image Stabilization, allows you to take handheld, long exposure-style shots without the need to brace your arms against a solid option to prevent blur.

The concept is nothing new. Older Huawei phones had something similar. However, advanced processing used here is designed to let you use it handheld. You needed to keep Huawei phones still for several seconds to get usable results before. Now you don't.

Huawei calls this AIS, or AI Image Stabilization.

We can thank the Kirin 970 processor's NPU for this. NPU stands for "neural processing unit" and it excels at processing image and sensor-related data quickly.

There's also something of a red herring in the Huawei P20 Pro's camera spec, though. Huawei claims sensitivity goes up to 102400, which is exceptionally high. Even shots from full-frame DSLRs look ropey at this sensitivity.

However, our interpretation isn't that Huawei means this sensitivity level is actually used in a traditional sense. It seems more likely the Huawei P20 Pro can operate at the ultra-low light levels that would normally require this sensitivity, when using the Super Night mode.

Sure enough, Huawei also says the camera takes decent photos at light levels as low as 1 lux, one lumen per square metre. This is ISO 102400 territory.

Actual ISO 102400 from a 1/1.7-inch sensor would have the noise and clarity of TV static.

Huawei claims the P20 Pro's sensitivity goes up to 102400 

Huawei claims the P20 Pro's sensitivity goes up to 102400 

Getting Computational

Huawei mixes traditional photographic terms with comparable results achieved using what it loves to call AI (artificial intelligence). Another manufacturer might simply call it advanced, fast processing.

It is borderline deceptive, but these smarts are seen throughout the camera setup, to great effect.

Every time you take a photo, they are used to enrich colour, improve skin tones and to merge the results of multiple sensors. It's used when you zoom in to 5x. It is also used by the front camera.

Like the iPhone X, the Huawei P20 Pro can emulate studio lighting effects. Here it's the selfie camera that gets the feature.

The 24-megapixel front sensor uses face mapping to make the effect "3D", rather than just a filter applied to a flat image. This face mapping also informs face enhancement, the skin-smoothing, feature-improving effect you've probably seen before.

Huawei uses "AI" smarts for focusing too. It only explicitly mentions this in reference to the P20 Pro's advanced object tracking, designed to keep the subject sharp as it moves across the frame. However, there are also serious smarts involved in juggling the P20 Pro's various focus techniques.

There's laser focus, phase detection, depth detection and contrast detection. The effectiveness of each varies depending on the scene and light conditions (and laser focus is an aid rather than a true focusing method).

There's also laser focus, phase detection, depth detection and contrast detection 

There's also laser focus, phase detection, depth detection and contrast detection 

P20 Pro sensor: Who makes this thing?

The Huawei P20 Pro camera is an intriguing mix of smart hardware and even smarter software. However, one of the most important questions remains a mystery. Who makes the 40MP sensor?

Sony and Samsung make the majority of CMOS image sensors for phones. Neither has any published data on a 40MP sensor. 

Neither do Omnivision or STMicroelectronics. Toshiba made the sensors for the Nokia 808 PureView and Lumia 1020, but it sold that division to Sony in late 2015 for $155 million, and now only produces cameras for industry purposes.

Who's behind the P20 Pro camera magic? We have no idea, but we'd like to know.