logoalt Hacker News

j2kuntoday at 6:50 PM0 repliesview on HN

The linked repository has demos you can run (though you have to install bazel), and some of the smaller models run inference in about a second, while the larger ones take minutes.

That said, there is a lot of ongoing work on GPU acceleration. Cf. the recent FHE-based CIFAR demo that runs in 200ms: https://sofar.belfortlabs.cloud/

Still maybe 1000x slower than cleartext, but progress!