Why metrics—and context—matter: How CISOs can measure and communicate cyber resilience

Cyberattacks in the digital supply chain are now some of the most common cyber incidents today, with many of the recent major breaches resulting from a single vulnerability. Because of the rapid pace and scale of these attacks, cyber leaders can no longer rely on static analyses of their environments, and must continuously assess cyber risk across their entire supply chain and vendor ecosystem. They must also produce quantitative metrics to measure their organizations’ dynamic risk in a standardized, easy-to-understand way. Here, we’ll discuss several metrics that cyber leaders can use to measure cyber risk and communicate with multiple stakeholders. 

Metrics that matter

As a CISO for one company and the Chief Trust Officer for another, I can probably understand—better than most—the fraught relationship some of my fellow cybersecurity practitioners have with security ratings. The origin of the concern around ratings is that when the measurement of cybersecurity resilience first came out in the third-party governance world, there were a lot of perceived anomalies that were associated with Internet-facing devices. 

For a long time, it was a binary: a device  was either connected to the enterprise’s network or not and configured a certain way or not. But the problem is that asset ownership is somewhat murky in most large enterprises. This means that third parties often have Internet-facing devices that aren’t necessarily associated with the enterprise or, if they are, they’re loosely associated. Security issues stemming from these devices can be difficult to fix, and—depending on their relationship with the third party—this reflects poorly on the cyber posture of the enterprise. 

Historically, this was a difficult problem to fix. But SecurityScorecard recently did something about it and developed security ratings exclusively for telecommunications, internet service providers, and cloud providers. Large enterprises, especially in this area, have long relied on large networks of third-party vendors and partners. This tailored approach to an industry with myriad nuances is one example of how security ratings have evolved since they first hit the market. Not only has their accuracy improved, but they’ve become more efficient, more industry-friendly, and more transparent. 

Don’t reveal too much   

Beyond security ratings, there are many other metrics for cyber leaders to lean on; and they vary depending on industry, organization, and priority. My philosophy is that the worst thing a CIO can do is to share every security metric and key performance indicator (KPI) with senior leadership. I say this because, in my experience, the CEO, other senior leaders, and board members don’t necessarily know what to measure at any point in time for determining the effectiveness of the cybersecurity posture and resilience across the enterprise. And, more crucially, it’s not really their responsibility to know that.

“…measuring something because it’s possible ultimately pollutes key performance indicators…the best KPIs measure output or outcomes that clearly determine the health of the process.”

As a result, my preference is for the CIO/CISO to curate and select which KPIs to share, and then to put those in a context that will be easily understood. For example: if the board and senior executives are principally concerned about third-party governance, I’d select the KPIs that are most relevant to supply chain hygiene, as well as the governance process that’s in place for supply chain management.

Keep in mind that measuring something because it’s possible ultimately pollutes key performance indicators for cybersecurity KPIs. KPIs are specific measures of health for a process, workflow or function with controls embedded. The best KPIs measure output or outcomes that clearly determine the health of the process.

Context matters

People remember stories more readily than they remember facts or metrics. When presenting to stakeholders whose backgrounds aren’t rooted in cybersecurity, it’s important to select specific KPIS and wrap them in a story. Here’s an example from a hard lesson I learned: One time years ago, I was the new CISO of a company, and I informed the CEO that the number of security vulnerabilities went up by 60% from one year to the next year. And of course, this CEO was aghast at this and asked me how the number went up by so much, so quickly. My point was that we had uncovered more vulnerabilities than we were originally aware of and were now able to get to their root causes and fix them. But, of course, I didn’t lead with that. I led with a metric instead of the story and context. 

Measuring behaviors

Cybersecurity leaders in both the public and private sectors need to ensure they have trustworthy, reliable data that measures cyber resilience and effectiveness. The first part of this equation is recognizing that measuring cyber resilience really comes down to measuring a set of behaviors enterprise-wide, at scale. And that’s fundamental to every cybersecurity program. New cyber regulations from the SEC, the EU (DORA), the White House, and beyond represent a shift away from decades-old voluntary compliance guidelines to a set of more aggressive regulatory approaches. 

The emphasis on measuring and communicating cybersecurity risk is growing. Security teams should establish repeatable processes and effective communication channels to enhance their organization’s safety, maintain trust, and bolster cyber resilience.

For more on these topics, listen to my podcast interview with Cybercrime Magazine Editor-in-Chief Steve Morgan.