logoalt Hacker News

0123456789ABCDE • today at 12:23 PM • 1 reply • view on HN

perhaps i'm not pilled enough for this conversation, but aren't you conflating two disparate concepts — model-weights and context-window?


Replies

kalcode • today at 3:49 PM

Yah I think I am conflating it. I was thinking using the ram more effectively to begin to squeeze more, one of those is context size. Less VRAM the less context you can handle, so there always a lot of tuning to find that sweet spot.

So my mind on that.