Programming, philosophy, pedaling.

PGP signatures on PyPI: worse than useless

May 21, 2023     Tags: cryptography, devblog, programming, python, rant    

This post is at least a year old.

TL;DR: A large number of PGP signatures on PyPI can’t be correlated to any well-known PGP key and, of the signatures that can be correlated, many are generated from weak keys or malformed certificates. The results suggest widespread misuse of GPG and other PGP implementations by Python packagers, with said misuse being encouraged by the PGP ecosystem’s poor defaults, opaque and user-hostile interfaces, and outright dangerous recommendations.


I’ve been sitting on this post for a few months, in part because of travel and in part because its (intended) scope was beginning to reflect PGP’s own fractal complexity.

The version that I’m publishing now has been significantly pared down to remove extended digressions on how bad PGP’s packet format is, all the different ways in which a signature or certificate packet can be broken, incorrectly bound, &c.

I’ve removed those things because I think the results, as present, are sufficient evidence for the actual claims I’d like to make, namely:

  1. That existing PGP signatures on PyPI serve no security purpose, and that all evidence points to nobody ever attempting to verify them;

  2. Even advanced technical communities, as a whole, largely fail to reduce PGP’s complexity and unnecessary agility into a reasonable and tractable subset.

And, just in case it needs to be said:

  1. This post isn’t intended to disparage PyPI: PyPI has done everything right, including purposely removing frontend support for PGP years ago.

  2. This post isn’t intended to disparage individual packagers and maintainers still uploading signatures to PyPI. I suspect that much of the ongoing signature uploading is a result of long-forgotten automation and, even when it isn’t: developers cannot be blamed for their misuse of obtuse tools. Security tools, especially cryptographic ones, are only as good as their least-informed1 and most distracted user.


PyPI has supported PGP signatures in some form or another for a very long time2.

To this date, PGP is still (minimally) supported: package uploaders can still sign for their package distributions and upload the resulting .asc to PyPI for inclusion in the index. The official uploading utility even supports invoking gpg directly via the --sign and --sign-with arguments!

To a novice Python programmer looking to publish their first package to PyPI, this might give the following impressions:

  1. That PGP offers secure and modern cryptographic primtives;
  2. That PyPI encourages users to upload PGP signatures or that doing so is best practice;
  3. That others expect PGP signatures, and that package adoption is (in part) predicated on supplying PGP signatures.

The first two are just wrong:

  1. PGP is an insecure and outdated ecosystem that hasn’t reflected cryptographic best practices in decades.

  2. PyPI’s support is vestigial in nature: signatures are not shown as part of the web interface, and are only obliquely referenced in the PEP 503 and JSON APIs.

The third is harder to immediately refute: PyPI still hosts signatures, after all. Absent any other information, it’s entirely possible that companies and end users are quietly and diligently verifying whatever signatures are present, using trust sets, tracking revoked and expired keys, and so forth.

Thus, my goal with this blog post:

  1. Determine how many signatures are on PyPI;
  2. Correlate those signatures to their signing keys;
  3. Analyze those signing keys for their practical value: their strength, liveness, &c.


Relatively early in the process I decided not to collect every single signature on PyPI, for two main reasons:

  1. Relevance: PyPI hosts many old package distributions, including distributions for Python 2.7 (and earlier!). Given that Python 2 has been EOL for over three years at this point, it didn’t feel relevant (or efficient) to retrieve large quantities of signatures that nobody is likely to ever try install the distributions for.

  2. Fairness: both PGP and Python have a lot of history, much of which predates modern understandings around cryptographic best practices. Given that, it didn’t feel fair to analyze extremely old signatures, especially if doing so would bias the statistics away from newer users who are doing more responsible things.

Given these considerations, I decided to limit my analysis to only signatures uploaded to PyPI on or after 2020-03-27. I chose that date somewhat arbitrarily3 while also satisfying a few constraints:

Actually retrieving the signatures was a multi-step process. To start, I used PyPI’s BigQuery dataset to give me some basic metadata on every distribution file with an associated signature:

SELECT name, version, filename, python_version, blake2_256_digest
FROM `bigquery-public-data.pypi.distribution_metadata`
WHERE has_signature
AND upload_time > TIMESTAMP("2020-03-27 00:00:00")

This produced 52900 distributions uploaded since 2020-03-27 for which PyPI also had a signature (subtract 1 for the CSV header):

$ wc -l inputs/dists-with-signatures.csv
52901 inputs/dists-with-signatures.csv

$ head -2 inputs/dists-with-signatures.csv

From here, I needed to retrieve each release distribution’s detached signature, i.e. the adjacent .asc URL in PyPI’s object storage.

I initially did this with the “conveyor” service, which turns PEP 491 names into URLs like so:


However, this was pretty lossy: for whatever reason4 my URLs were slightly off about 20% of the time, resulting in lots of missed signatures. I eventually realized that the BigQuery dataset also includes the Blake2 digest for each distribution, meaning that I could use the actual package URLs instead:


…and this was perfectly reliable.

From here, I wanted to figure out (roughly) how many unique keys produced these ~50k signatures. I decided to use PGPy5 for that; excerpted from dists-by-keyid.py:

sig = pgpy.PGPSignature.from_blob(sig_resp.content)
    # https://github.com/SecurityInnovation/PGPy/issues/433
except AttributeError:
    print("barf: couldn't get signer, probably ancient", file=sys.stderr)
    _KEY_ID_MAP["<invalid signer>"].append(rec)


This left me with a big map of PGP key IDs6 to a list of distributions signed by them, including 26 distributions whose signatures PGPy couldn’t parse:

Package name Distribution count
agraph-python 2
excerpt-html 4
lektor-index-pages 6
lektor-expression-type 2
lektor-git-timestamp 2
lektor-datetime-helpers 3
lektor-limit-dependencies 2
lektorlib 2
lektor-polymorphic-type 3

This is a tiny failure (26 distributions out of 52900, or roughly 0.5%), but it sets the tone for the rest of the post.

Apart from these 26 failures, the remaining 52874 signatures were produced from 1067 “unique”7 PGP keys.


At this point, I had 1067 unique key IDs, each of which needed to be retrieved from a keyserver.

My expectation was that this wouldn’t be a significant challenge, despite the widely publicized implosion of the SKS keyserver network back in 2018: there are still a few major keyservers running, and package authors pushing to PyPI should have the presence of mind to upload their keys. Right?

Pictured: your author immediately before trying to retrieve PGP keys in 2023.

Wrong. Of the 1067 keys IDs collected through signatures on PyPI, a full 308 (or roughly 29%) had no publicly discoverable key on the major remaining keyservers. In other words: roughly 1/3rd of all signatures added to PyPI since 2020 are bound to keys that aren’t discoverable by the PGP ecosystem’s own tooling. They might exist, hidden on personal domains and documentation pages, but, for all intents and purposes, these 29% of keys are useless8.

So, our first graphic of the post: discoverable keys versus undiscoverable ones:

Pictured: a very normal and healthy signing ecosystem.

That left 759 discovered keys to actually audit. To keep things simple9, I limited my analysis to just the following considerations:

If that seems like a limited analysis, it’s because it is: there are too many ways to produce a weirdly shaped PGP certificate and/or key packet sequence, and the existing tooling (things like pgpdump and pgp --with-colons) weren’t up to the task.

Instead, I wrote a little tool (pgpkeydump) to give me machine-readable dumps of PGP keys11, and then wrapped it in a bulk auditing script that does some basic statistics on the results.

To summarize the results:

Then, on the algorithm and parameter sides12:

Primary keys:

Key type Count
RSA-4096 497
RSA-2048 127
RSA-3072 45
DSA-1024 40
EdDSA 35
DSA-3072 7
DSA-2048 4
NIST P-521 1
RSA-4064 1
RSA-4032 1

“Effective”13 keys:

RSA-4096 471
RSA-2048 151
RSA-3072 47
EdDSA 43
DSA-1024 31
DSA-3072 7
DSA-2048 5
NIST P-521 1
brainpoolP512r1 1
RSA-4032 1

Or again, as pretty charts:

First, the “good” parts:

  1. While normally a bad choice, RSA is literally the best you can do in terms of standard14 asymmetric signing algorithms in PGP. Over two thirds of keys used to sign on PyPI are using it, and they’re using reasonable15 key sizes (4096 and 3072).

Then, the meh:

  1. A sizeable minority (20% of effective keys, and 17% of primary keys) are RSA-2048. NIST considers RSA-2048 to be equivalent to roughly 112 bits of security16, and does not recommend its use on data that’s expected to have a security life of 15 years…starting in 2015. That means that PyPI-hosted signatures against RSA-2048 keys have roughly 7 years of “shelf life” in them. Version turnover in packaging ecosystems has accelerated over the last decade; let’s hope that applies here too!

  2. Some enterprising people are on the “bleeding edge”: they’re using EdDSA and a few different ECDSA curves. It’s hard to say whether this is good or bad: it’s good in the sense that these are almost certainly better than anything offered by strictly RFC 4880 PGP implementations, but pointless in the sense that support for verifying these signatures is limited17 to just a few clients. It’s also probably pointlessly slow (for P-521 and brainpoolP512r1 in particular).

And finally, the insane:


To summarize: of just the PGP signatures uploaded to PyPI in the last three years:

By all rights, these numbers represent the best possible case for PGP signatures on PyPI. Expanding the audit to 2015 or even earlier would likely reveal far worse practices.

In one sense, none of this is a problem: the breadth and depth of issues here suggests that nobody (thankfully!) is actually relying on these signatures, and the continued presence of new signatures on PyPI is primarily a vestige of forgotten automation and outdated tutorials.

On the other hand, these results present a strong case against attempting to “rehabilitate” PGP signatures for PyPI, or any other packaging ecosystem: all evidence points to end users (i.e., signers) being unable19 to distinguish between the “good” and “bad” parts of PGP, much less use them at all (e.g. keyservers).

So, for final conclusions:

As with previous posts, I’ve tried to make my steps and data reproducible, and have checked them all into this repo. I welcome any discoveries of mistakes I’ve made, as well as any attempts to improve the overall detail or fidelity of the results!

  1. In a domain-specific sense: nobody should have to be an expert in compilers to enable basic security mitigations, and nobody should have to be an expert in cryptographic protocol design to generate a good signature. 

  2. It’s hard to tell exactly how long, but it’s potentially as old as PyPI itself: 23 year old design threads mention PGP as an early consideration. 

  3. It’s exactly three years before before the day I began this post. 

  4. I was too lazy to debug this, but it was probably because I was assuming that all distribution URLs were wheel-like, when many were source distributions. Update: Ee has informed me that this was probably because of a lack of normalization: conveyor doesn’t normalize package or version names on either end. 

  5. As the snippet suggests, this was probably a mistake: PGPy is very lightly maintained and appears the win the jackpot in terms of simultaneously being incompatible with old PGP signatures and lagging behind the rest of the PGP ecosystem. 

  6. As in, the 32 byte/8 hexdigit key IDs that everyone is used to. You know, the ones that are trivially collidable and have been for years. 

  7. PGP has both keys and “subkeys,” and the relationships between them are pointlessly malleable. Given that, the number is really 1067 unique key IDs; it’s impossible to say how many unique containing certificates or representations of each key have been made over the years. 

  8. I’m also giving the PGP ecosystem a break here, by acting as if a key’s presence on a keyserver somehow makes it trustworthy. This isn’t true: you still need to have a reason to trust the key, which schemes like the web of trust and strong set were meant (and failed) to provide. 

  9. Things were originally not simple: I started out by writing a full PGP certificate and key linter, 

  10. A PGP certificate that doesn’t contain a binding signature is effectively not a certificate, since it contains no positive evidence that someone actually possesses the private half of the key. 

  11. Really PGP “certificates” or “sequences of packets resembling PGP certificates,” but nobody uses these terms consistently in the PGP ecosystem. 

  12. The eagle eyed might notice that the total key count here is off by one: 758 instead of 759. That’s because there’s one key ID, CD6F6C3E0A50F73B, that doesn’t even match the key returned by the keyserver! I have no clue how this happened, and I can’t be bothered to figure out. 

  13. “Effective” means the signing key, which can either be the primary key or a subkey. I audited both (when different), under the operating theory that it’s bad to have a strong subkey bound to a weak primary key (cf. a strong TLS certificate issued by a weak CA). 

  14. Meaning RFC 4880 compliant, not the miscellaneous other optional RFCs that various implementations may or may not choose to support. 

  15. In terms of cryptographic safety margins, not representation size. Representation wise, both RSA-3072 and RSA-4096 are ridiculously large and unwieldy compared to EC keys with similar or stronger margins. 

  16. Which itself is discouraged: NIST’s own recommendation is to prefer a minimum of 128 bits of security, which would correspond (roughly) to RSA-3072. 

  17. And, if your use of PGP involves an incompatible subset, you might as well just do things right and drop PGP entirely. 

  18. And I didn’t bother checking. 

  19. Which, again, is not their fault: the system itself bears complete responsibility. 

Discussions: Reddit Mastodon Bluesky