You should be able to run smaller models on an M1. I'm testing this in about 10mins
how did it go wsgeorge? is there like a 10 second pause between each word when running on a Mac? I thought I could only run 8b models on it from what I remember last year and even those were super slow!
alt Hacker News
how did it go wsgeorge? is there like a 10 second pause between each word when running on a Mac? I thought I could only run 8b models on it from what I remember last year and even those were super slow!