alt Hacker NewsOh, is that how it works? So, when somebody says a model is running at X tokens per second, it means that the thinking process is running at that, and output tokens are much lower then? Thanks to the explanation.
Oh, is that how it works? So, when somebody says a model is running at X tokens per second, it means that the thinking process is running at that, and output tokens are much lower then? Thanks to the explanation.
It’s all just tokens to the model. Whether it’s thinking tokens or output tokens, they take the same amount of computation to produce. The only difference is whether the token is displayed to the user.