It does affect the throughput for an individual user because you need all output tokens up to n to generate output token n+1
:facepalm: - That’s not how that works.
alt Hacker News
:facepalm: - That’s not how that works.