Interesting concept, but is it going to age well with context sizes of models are changing all the time (growing, mostly)?
max context sizes are probably going to go up, but smaller contexts will always be cheaper/more-efficient than larger ones
alt Hacker News
max context sizes are probably going to go up, but smaller contexts will always be cheaper/more-efficient than larger ones