alt Hacker NewsAt least for the sake of this demo, the costs are actually pretty small, especially given that many of the ... even at $1.75/1M tokens you'd have to have 2 megabytes of text across all users before this even broke $1...
At least for the sake of this demo, the costs are actually pretty small, especially given that many of the ... even at $1.75/1M tokens you'd have to have 2 megabytes of text across all users before this even broke $1...
The unit price looks low but the cost driver in these apps is usually the context window. You have to re-send the full history every turn to maintain state, so you end up paying for the same tokens dozens of times. By the end of a long session the cost per interaction is significantly higher than at the start since you're re-processing the entire game log for every new command.