logoalt Hacker News

osamagirl69 • today at 5:23 PM • 0 replies • view on HN

The chart of the actual tok/s vs concurrency is at the end. For a single user you get 107tok/s, which is about 2-3x as fast as you would get on 4x DGX spark.

The 607 tok/s is total tok/s across the 16 users, so a bit over 35tok/s per user.