logoalt Hacker News

storystarlingyesterday at 11:20 PM1 replyview on HN

The unit price looks low but the cost driver in these apps is usually the context window. You have to re-send the full history every turn to maintain state, so you end up paying for the same tokens dozens of times. By the end of a long session the cost per interaction is significantly higher than at the start since you're re-processing the entire game log for every new command.


Replies

selcukatoday at 1:55 AM

Cached tokens are cheaper. Also there are ways to compress the context (not sure if this game employs any of those techniques).