Mar 20, 2024 Tags: oss, security
About 15 months ago, I posted a rant about misaligned incentives in the vulnerability triage and classification ecosystem1, with particular attention given to low-impact, high-noise categories like ReDoS.
Since then, I’ve had more time to think about the overall role of the CVE system and the parties that participate in it. I haven’t changed my mind about the overall (limited) value of categories like ReDoS, but I do have new thoughts (brought about by recent developments) that color my overall view of vulnerability reporting, triaging, and the incentives that continue to drive money and engineering effort into the space.
This post is an attempt to flesh out those thoughts2. No particular degree of cogency is promised.
When I wrote my original post rant, one of my assumptions was a
common understanding of the purpose of the CVE system: I incorrectly understood
(and believed everybody to understand) identifier
assignment to be a normative claim about the existence of a vulnerability
and, consequently, a normative claim about that vulnerability’s severity3.
From that assumption, my objection was a technical one: I did not (and do not) believe that just the presence of a potentially intensive regular expression constitutes a vulnerability.
Over the last year, I’ve learned that this is not the only way people (including serious security people who I respect!) understand vulnerability identifiers.
Another understanding of CVEs IDs, one with substantial historical, is as merely stable identifiers. In this view, CVE IDs and other identifiers do not intrinsically represent a claim about a vulnerability’s existence, but are instead “just” unique handles that enable alerting, minimize confusion, and reduce duplicated effort among busy and distracted engineers.
Under this understanding, vulnerability feeds are not responsible for the factual contents of their reports: the responsibility of the feed is solely to uniquely identify and distribute a common data format, one that individual consuming parties are entirely responsible for interpreting and assigning importance to.
This interpretation has desirable justifying properties:
The CVE system is, theoretically, simple to maintain: it has only minimal curatorial responsibilities, boiling down to (1) ensuring the uniqueness of identifiers, (2) preventing outright spam4, and (3) performing deduplication and/or mapping against related identifiers. Each of these, while not free, is substantially cheaper to perform that full vulnerability triage.
The CVE system cannot mislead vendors about severity: vendors must make their own judgement calls, treating CVE as a “firehose” rather than a pre-qualified source of truth (which, again, would be very expensive).
The CVE system protects security researchers: because CVE IDs are “just” unique identifiers, researchers can collaborate and share their work without being prematurely cut out of the process by hostile vendors or unsympathetic engineering teams.
I like these properties. However, in 2024, they largely ring hollow to me:
Empirically, the CVE system has not been simple to maintain: the mechanisms in place for CNA and CVE ID allocation and management do not support current usage patterns, as evidenced by a growing community expectation that CVEs issued by MITRE cannot be meaningfully contested and NVD’s own sitewide warning banner:
CVEs are almost universally accompanied by vulnerability scoring schemes like CVSS, schemes that are almost universally agreed to be useless in the limited context of a CVE. Official mouthpieces and disseminators of CVEs (like NVD) choose to render CVSS scores prominently next to CVEs, enabling confusion and conflation between the two5.
Security researchers don’t need the same kind of protections that they did 25 years ago: security has legal and geopolitical aspects that prevent it from being systematically ignored6 in the way that it once was, and the engineering community has culturally adapted7 to the idea that security is a part of the development lifecycle.
In effect, the CVE system benefits from a position of strategic ambiguity: it can be a dumb source of unique identifiers when its supporters (and some detractors8) need it to be, and it can also be a rich source of severity metadata when bug bounty programs, reputation seekers, and security vendors so need it to be.
The CVE system’s participants don’t universally encourage this ambiguity, but none appear to discourage it either. After all, there are very few incentives in place for NVD, MITRE, AISuppySecTrustShieldly™, &c. to dispel any ambiguity here: ambiguity maintains industry and regulatory interest in the problem space, ensuring ongoing funding and engagement9 in what ultimately boils down to a unique identifier.
So, who’s responsible?
In my original post, I split the responsibility 3 ways, suggesting:
I believe that these responsibilities largely hold true and, critically, have paid off over the past year: there has been a growth in pushback against the incentive structures that got us into this mess, including by individual maintainers.
At the same time, it’s no longer clear to me that there is a sufficient common understanding of the purpose of the CVE system against which to justify systematic changes to how we approach vulnerability identification, triage, curation, &c.
In other words: so long as the CVE system’s participants voluntarily maintain the system’s strategic ambiguity (and have a financial interest in doing so), things will not systematically improve.
One way to “attack” strategic ambiguity is to force it to reveal its hand by contriving a situation that cannot be made simultaneously stable and ambiguous.
This is the approach that Linux’s new CNA program appears to be taking:
Note, due to the layer at which the Linux kernel is in a system, almost any bug might be exploitable to compromise the security of the kernel, but the possibility of exploitation is often not evident when the bug is fixed. Because of this, the CVE assignment team are overly cautious and assign CVE numbers to any bugfix that they identify. This explains the seemingly large number of CVEs that are issued by the Linux kernel team.
In (rightfully!) assigning a CVE to most bugfixes, the Linux kernel CNA has forced security vendors out of a state of comfortable, stable ambiguity: no curation can be assumed, downstream consumers are (rightfully!) annoyed at the deluge of irrelevant reports passed on to them that should have been filtered by their vendors.
By the reasoning in my previous post, I ought to consider this a disaster: not unlike ReDoS reports, this flood from the Linux CNA risks security and alert fatigue. But instead, I’m (cautiously) optimistic: unlike ReDoS and other largely chaff categories, a rise in volume here is not noise, and has the potential to counter the incentives that brought us to the this state.
That leaves us in a funny state, one where things will get worse for good reasons before they can possibly get better. That’s not great, but it does leave open room for systematic improvement where none seemed possible before. At least, that’s how I’m thinking about it now.
…and nascent for-profit industry. ↩
So that I can perhaps revisit them again. ↩
At the very least, in the most basic sense of “you wouldn’t call it a vulnerability if it wasn’t even minimally severe.” ↩
Of the “Nigerian prince” variety, not the “ReDoS vulnerability in black
” variety. ↩
To the extent that “CVE score” is an intelligible thing that security people will say, despite CVE IDs not having their own scores. ↩
Which is not to say that security is not still ignored all the time, only that companies now (in part thanks the NIST/NVD!) find it much harder to sweep serious breaches under the rug. ↩
“Shift-left,” “DevSecOps,” blah blah. ↩
In the sense that, as a mere identifier, a CVE ID is never itself a cause for concern. ↩
To be clear, I don’t begrudge agencies, FFRDCs, startups, &c. this: a sense of urgency is often the only way to obtain essential funding. But that doesn’t make the incentive structure itself good. ↩