logoalt Hacker News

smithzayesterday at 8:52 PM1 replyview on HN

pickle files are usually the limiter here. I would be surprised if it can handle pickle files since it relies so much on runtime LUTs of the objects and arbitrary object definitions. This usually doesn't work in other use cases such as swig or cython either IIRC.


Replies

cdavidyesterday at 9:27 PM

For NumPy/Pytorch, the C API is much bigger issue than pickle. I have not looked at the architecture of this, but given it uses its own IR + replaces ref counting w/ a GC, I am assuming it does not have C API compatibility.