logoalt Hacker News

PyPI Blog: Releases now reject new files after 14 days

73 pointsby mikethemanlast Wednesday at 2:20 PM37 commentsview on HN

Comments

nneonneotoday at 3:13 PM

The remaining risk now is that a patient, malicious actor could put out a new, clean source-only release, wait for ~7 days for people to decide it's safe and update to that version (and pass typical update delay controls), and then attach a bunch of malicious binary wheels. 14 days still seems to be too long.

Of course, this is already miles better than the current state of affairs where an old but popular package could become an infection vector at any time.

yladiztoday at 12:21 PM

I’m a bit surprised this is possible in the first place. I get that you might not be able to upload everything in one go, but it feels like you should “start” and “finish” a release in that case, and once it’s finished you can’t modify it.

I guess the use case is that you might want to build a wheel for an older release for a newer version of Python?

show 1 reply
llg-312gtoday at 6:01 PM

Python packaging, the most convoluted way of creating simple zip files imaginable.

The tool fragmentation is insane, the demand to create "source distributions" was maybe funny in 2002 but just a hindrance now.

Packages no longer build since distutils was ripped out and upstream replaced it with meson etc.

Since building from source no longer works, which is profitable for third party vendors like Conda, "wheels" are uploaded. And they cannot be built on the server since the whole "scientific" ecosystem is perpetually broken. And they are separate artifacts, leading to the above problem.

Shipping checksummed tar archives is of course it not possible, that would hurt the income streams of the package profiteers.

skinfaxitoday at 1:56 PM

14 days is still too long if you ask me. Releases should be immutable.

show 1 reply
firesteelraintoday at 1:03 PM

This seems like common sense configuration management 101. If I download v1.2 and it’s been published then it should be considered released and not modifiable. With exceptions for ‘dev’ releases of course. I have never published anything on PyPI but I would expect there is a publish button and finalize (?) optional button that if not checked after 14 days makes it final ?

edelbittertoday at 12:25 PM

> To quantify how disruptive this change would be to existing workflows, the PyPI database was queried for projects that have published new files to old releases

While this may quantify how disruptive the change would be to those projects that are able to and do upload additional binaries to PyPI later, it fails to quantify how many projects already completely circumvent this block before it is even introduced.

e.g. If you tell pip to install from source.. the result may already be that you install a binary that PyPI never saw. A common hack for dealing with NVidia internals, which can explode into a large CUDA major version x GPU arch x platform x implementation x python_version cartesian product. The "extras" mechanism is not quite sufficient to model such combinations.

sample code: https://github.com/Dao-AILab/causal-conv1d/blob/4f6ae4e26ae5... https://pypi.org/project/causal-conv1d/

PhilipRomantoday at 12:21 PM

There seems to be a severe lack of hash pinning in "modern" software ecosystems. We figured out how to do it 20+ years ago with Git bringing hash addressed storage to the masses. Coming from a different background it was very surprising for me to see things like docker images, packages and github actions being updated at the whim of upstream registry. I much prefer the philosophy where builds are fully offline and predictable, even if not fully reproducible.

show 4 replies
kapilvttoday at 3:43 PM

The other consideration that would be useful is an explicit api for a developer to freeze the release, to prevent new file upload.

ascendantlogictoday at 3:32 PM

Kinda curious why releases just aren't fully immutable? Sane semver would dictate any update should at least be a new patch release.

show 1 reply
crabbonetoday at 6:04 PM

It's kind of hilarious how everything that has to do with Python is so obviously wrong, with the obviously right way of doing things being right there, on the service, having been around since before Python even existed... and yet, Python will fail spectacularly every time.

If you ever used Maven, NPM or... I can't think about any other tool that doesn't automatically check checksums and signatures. Any Linux package manager ever used... Python's Wheel format has provisions for checksums and signatures! But they aren't checked.

Instead Python gets absurdly ineffective workarounds that will probably inconvenience a few developers and will do zilch for users.

lexicalitytoday at 12:51 PM

14 days still seems way too long to me. As a user I thought releases on pypi were immutable!

show 2 replies