alt Hacker NewsIt doesn't make sense at all. So as a user how do you choose which model to use? There could be 3824 models to choose from. The browser might as well set one as default, and we all know how that goes (see: search engine).
Not to mention many other UX questions the come with this, most importantly, how unusable these local models are on regular 3-year old laptops that are constrained in RAM, GPU/CPU capability and likely disk space despite what enthusiasts say here. (They have a Macbook Pro with 32+GB of RAM, reports it works great with xyz model -- fine -- but somehow thinks it works for everyone and local models are the future.)
Replies
> It doesn't make sense at all. So as a user how do you choose which model to use? There could be 3824 models to choose from. The browser might as well set one as default, and we all know how that goes (see: search engine).
...what's the exact problem here? Believe it or not, most non-tech-savvy users use the search engine just fine.
The Chrome model requires either "16 GB of RAM or more and 4 CPU cores or more" or "Strictly more than 4 GB of VRAM", and "22 GB of free space" (it uses around 4.4GB but it doesn't want to use the remaining free space).
The model is pretty slow on my M4 Pro mac.
The API allows the browser to use a cloud service instead, but then privacy is lower. So, more privacy for the rich.