A medium prompt = 1 million tokens?
How many times have you had a model start compacting already before getting back to you? Most have 1 million context window. It's happened to me occasionally
How many times have you had a model start compacting already before getting back to you? Most have 1 million context window. It's happened to me occasionally