'We have seen some nasty things': Here's what a VPN audit really involves

Graphic of laptop testing
(Image credit: Shutterstock)

In the past, users were asked to blindly trust the credibility, safety and privacy of their VPN service. But in a time where the trustworthiness of VPNs is paramount to their users, more and more providers are backing up their privacy promises with deeds rather than empty words. 

Performed by an experienced independent third-party company, a VPN audit is the where one or several aspects of the software - such as its no-logs policy or app infrastructures - are tested. The more areas examined and the more access granted, the more reliable the audit results.

But, what guarantees can these audits give to VPN users? And, what happens when serious bugs and false security claims come to light?

We asked these and other queries to the founder of Cure53, Dr. Mario Heiderich - the head of the cybersecurity firm behind the audits of some of the top VPN providers around right now like Surfshark and ExpressVPN. Here's what he said.

 Mario Heiderich photo
Dr. Mario Heiderich

With past experiences as a data security developer and researcher under his belt, Mario is now using his knowledge to lead the Cure53 team. Over the years, he spread his expertise through conference talks and academic papers. Since 2019, he is also Chair for Network and Data Security at the Ruhr-University Bochum.

When was Cure53 founded and what was the idea behind it?

In 2007, starting as a one-person project (me) and then organically growing over time. Then, once more folks joined the team, it was driven for by the idea of "how should a company be where I myself would love to work" - and also, how happy team members make for better penetration tests.

We grew from a very small boutique outlet to a slightly less small boutique outlet. First only covering web application security, now being able to offer everything from bare-metal server to complex single-page application, cryptography reviews, infrastructure and network security, VPN, desktop software, C/C++, large scale source code audits, etc.

When did you begin working with VPN providers?

In 2016, when Tunnelbear reached out to us for a full, large scale system-wide audit.

Why are VPN independent audits important in today's market?

Because some players on the market offer serious products and services that intend to excel at security and privacy - and others just claim that and are more of a shady nature.

We have seen some nasty things during our audits. And there's a lot of reports that didn't go public because, well, the audits unveiled those nasty things quite accurately.

We 'fire' clients when we notice things are probably going sideways and that an offering doesn't hold what it promises (with the maintainer's knowing and accepting this for their own benefit, of course) and especially with VPN providers that has happened quite a few times in the past.

Could you give examples of the untoward things you have seen?

One example would be a provider who claims that they have a strict no-log policy and advertises publicly how privacy-friendly they are. Turns out, they log highly-sensitive info in three different places, act very defensively when we point that out, claim that this is information they need to make the service work (which is wrong) and then go into full drama mode when we refuse to remove that part from the report.

Another example would be a software client that bundles the Chrome browser using a very, very specific version (for no apparent reason) and it executes user-controlled JavaScript.

Turns out that this Chrome version can be attacked with a publicly available Chrome exploit and could be seen as a back-door to get full code execution on any users system whenever the provider (or anyone else who finds the issue) feels like it.

These findings are seemingly less frequent, but the reasons for that are unclear. Maybe providers get better? Maybe the shady ones don't request audits anymore? Hard to tell.

Cure53 homepage

(Image credit: Cure53)

What areas of a VPN provider's service can you audit and how?

We can audit:

- Mobile apps

- Desktop apps

- Browser extensions

- Servers, networks & infrastructures

- Architectures

- Web applications

- Cryptographic designs & implementations

- Payment processes

- Third party integrations

- Build chains and CI/CD

The way we do that really depends on what we actually look at from the above as well as on the scope of the work.

Which are the biggest challenges of this process?

Sometimes it's a challenge to convince the customer that their no-log audits make no sense, and that we cannot work using a threat model where the provider itself is a possible adversary.

The rest is usually like all other tests, smaller obstacles here or there but usually smooth.

What guarantees can a Cure53 audited VPN service give to its users?

Only one, we try our best. We try to find as many bugs as we can with the time and budget we have and do our very best to get ideal coverage and an objective verdict - and also make sure that the bugs we spot get fixed properly.

We're only humans and can overlook things and we know that, but we try not to overlook anything and be thorough and comprehensive.

Are there examples of your work bringing some substantial changes within a company?

Yes, but I'm not sure if I can tell any details given we are usually under NDA (non-disclosure agreement). But we had experiences where our work had tremendous impact on company (security) culture, development processes and most importantly awareness levels.

What could Cure53 do in case a VPN provider decides not to release a bad audit? Has this ever happened?

Yes, that did happen. Actually it just recently happened again. And well, if that is the case, we don't judge - I mean, we do but you know, it's in the end up to the company to decide.

If they want to publish, excellent - we support that every time it happens. But if not, then we're not the ones to add pressure, it's not our call to make.

Read more: Why your VPN provider should be audited

Chiara Castro
Senior Staff Writer

Chiara is a multimedia journalist committed to covering stories to help promote the rights and denounce the abuses of the digital side of life—wherever cybersecurity, markets and politics tangle up. She mainly writes news, interviews and analysis on data privacy, online censorship, digital rights, cybercrime, and security software, with a special focus on VPNs, for TechRadar Pro, TechRadar and Tom’s Guide. Got a story, tip-off or something tech-interesting to say? Reach out to chiara.castro@futurenet.com