logoalt Hacker News

nextaccounticlast Tuesday at 6:15 PM1 replyview on HN

> Most developers don't realize that standard PyTorch files are just Zip archives containing Python Pickle bytecode.

This is outrageous. Why not deprecate this cursed format and use something from the data frame community? Like, Parquet or something

Actually almost any binary format is better than this


Replies

tennysontlast Tuesday at 6:27 PM

Pickle files are probably still useful saving exploratory work, collaborating inside a company, and use inside a pipeline.

Safetensors is supposed to be the successor for distribution. I believe that it's the "safe" subset of pickle's data format.

show 1 reply