Security data lakes are key when strengthening cybersecurity

Security padlock on top of digital code numbers
Image Credit: Shutterstock (Image credit: Shutterstock)

Security leaders rely on data to keep their businesses safe and drive accountability. For instance, security data can help them understand which software teams are introducing the most vulnerabilities and how quickly their remediation teams can roll out critical patches.

Yet most CISOs struggle to access such data, due to the fact it sits across disparate systems and tools, stopping them from gaining a holistic picture. Without accessing these insights and bringing them together into one unified location, it’s virtually impossible for security leaders to understand crucial insights, which could be putting sensitive data at risk.

The solution? Security leaders and CISOs should utilize data lakes as an architecture to consolidate security data and implement a level of accountability that previously wasn’t possible. Security data lakes separate storage from compute, which makes it cost-effective to store security data at scale for a longer period. They also make security data part of a company’s general-purpose analytics platform, providing additional context and delivering insights via standard reporting tools. It’s high time security leaders take note and realize the benefits of security data lakes, and how they can help drive accountability across an organization.

Mario Duarte

VP of Security at Snowflake.

Holding vendors to account

Most companies select and evaluate security vendors based on simple criteria, like whether they support certain data sources and applications. A lack of information keeps decision-makers from evaluating vendors on more meaningful factors like threat detection performance or vulnerability prioritization accuracy.

Security data lakes allow businesses to identify gaps between the insights vendors provide and what the organization actually experiences. Analyzing data from the ticketing system, for instance, allows security teams to see how many threats detected by a vendor were false positives, or how many vulnerability findings turned out to be irrelevant.

A security product may work great in one company’s environment, but less well in another. If a business can measure performance across the metrics that matter to them, they can work with their vendor to help them improve — or determine that they need a better tool.

Identifying potential flaws

If remediation teams aren’t addressing vulnerabilities quickly enough on a consistent basis, access to historical data helps uncover those problems and identify processes that may need updating to help them work more effectively. Maybe workflows need to be adjusted, for example, or the team needs to be restructured to meet its SLAs.

A security data lake allows security teams to apply context at query time from non-security sources. For example, it can combine termination data provided by HR with security access policies to flag when an employee has an active user ID after they’ve left the company. Security teams can also correlate data about awareness training, phishing exercises, and actual malware cases to show how departments that don’t complete training are at greater risk of compromise.

A holistic picture

When teams are shipping new components into an organization's IT infrastructure, a security data lake can help track where vulnerabilities are consistently coming from the same groups — whether that’s developers, SREs, or some other entity. This kind of insight is difficult to achieve when data is spread across multiple tools and stored for short periods of time. With quantified metrics backed by data, security teams can fulfill their role in a shared responsibility model.

A secure future

It’s important to recognize that security teams are there to help employees and safeguard against potentially damaging security breaches. Their job isn’t to call out individuals for their own mistakes, but rather to ensure sensitive company data is kept safe. Without the right data to guide decision-making and hold teams accountable, security leaders are often working in the dark and will struggle to see the full threat landscape. As the saying goes ‘you can’t manage what you can’t measure’, and this is highly applicable to security teams.

With threats becoming increasingly prominent, and security regulations becoming ever more complex, there has never been a better time to reinforce security standards and unify data to ensure businesses are doing everything they can to avert risks. Security data lakes are a tangible solution and an ideal way of holding teams, individuals and companies to account.

We've featured the best online cybersecurity courses.

This article was produced as part of TechRadarPro's Expert Insights channel where we feature the best and brightest minds in the technology industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here:

Mario Duarte is the VP of Security at Snowflake. He has 20 years of experience as a security professional working across the retail, healthcare, and financial sectors.