alt Hacker NewsA gigabyte is a lot of memory. Even the largest context windows are a small fraction of that with any sane engineering discipline.
A gigabyte is a lot of memory. Even the largest context windows are a small fraction of that with any sane engineering discipline.
For each LLM interaction they likely have bunch of thoughts traces, tool calls, etc, which don't go to context, but still can be retrieved.
But I obviously don't know for sure.