alt Hacker NewsDo you need 450k context?
I usually keep under 200k if I can help it for Claude/ open ai
And lower locally on my Mac to save on prefill.
Do you need 450k context?
I usually keep under 200k if I can help it for Claude/ open ai
And lower locally on my Mac to save on prefill.
No, also because that would mean I have to read up on rope scaling or whatever to go beyond the default 256k context, which so far I’ve avoided. But you can allocate it as 220k context for 2 parallel Pi workers, that really speeds things up! Or as ~8x 50k context for dedicated parallel document processing.