alt Hacker NewsIf Chrome has the #optimization-guide-on-device-model and #prompt-api-for-gemini-nano flags enabled, either because it's part of some Origin Trial / Early Stable Release or something, then web pages will have access to the new Prompt API which allows any webpage to initiate the (one-time) download of the ~2.7 GiB CPU or ~4.0 GiB GPU model using LanguageModel.create()
https://developer.chrome.com/docs/ai/prompt-api
When Chrome 148 releases tomorrow, this will be the default behaviour on desktop.
To download, it should check for 22 GiB free disk space on the volume where your Chrome data dir is, and at least double the model size of free space in your tmp dir.
Replies
Those two (and more) exist in chrome://flags in Chrome 147. I'm disabling them now, with the expectation that will prevent the new default.
One option I'm leaving as default is "Use LiteRT-LM runtime for on-device model service inference." Any comment on that?
It is a small model, so what utility can I / Google expect from it? What is the on-board model used for?