Even with 95% cache hit, and on cheapest chinese model, it would still be in tens of thousands of dollars (I assume that of 500B tokens at least 10B would be in "output").
If we switch gears to Kimi K* then if would very quickly escalate in hundred thousands USD.
alt Hacker News
Even with 95% cache hit, and on cheapest chinese model, it would still be in tens of thousands of dollars (I assume that of 500B tokens at least 10B would be in "output"). If we switch gears to Kimi K* then if would very quickly escalate in hundred thousands USD.