alt Hacker NewsThis is really cool! I'd like to understand how you're dealing with the costs. Since you're using an LLM and this is getting viral on HN, aren't API costs getting higher? I'm working on some LLM project myself and trying to figure out how to make the demo accessible without spending too much.
Great job.
At least for the sake of this demo, the costs are actually pretty small, especially given that many of the ... even at $1.75/1M tokens you'd have to have 2 megabytes of text across all users before this even broke $1...