logoalt Hacker News

jgalt212today at 9:12 PM1 replyview on HN

A medium prompt = 1 million tokens?


Replies

RugnirVikingtoday at 9:53 PM

How many times have you had a model start compacting already before getting back to you? Most have 1 million context window. It's happened to me occasionally

show 1 reply