> This is an excellent use case for completely local, small model inference, yet for inexplicable reasons Mozilla wants to normalize uploading your entire private browsing history to a cloud.

I certainly agree that this option would be nice to have, but running firefox without an ad blocker on the modern web will ALREADY spin up my fans just rendering all the garbage, and I have a decent laptop.

You can't expect the median Firefox-running device to be able to run a capable enough local model effectively.

I am not trying to defend Mozilla doing this, and I don't support sending data to cloud based services like this in a way that users won't understand.

But I also think that the state of the art in small LLM and user device capabilities aren't there yet to put a "good enough to be actually useful" local-only LLM as a prepackaged thing in a mass market distributed browser.

You don't want a browser that takes 10GB of extra RAM (on top of the memory hog that is having just 3 or 4 complex tabs open on its own already) and pegs your CPU at 99% usage for minutes at a time. And not in an era when mass market consumer laptops are still commonly 8GB or 16GB of total system RAM. Many of those with integrated-into-CPU onboard graphics (eg: not a gaming laptop with a discrete GPU on the PCI-E bus).

It'll be a catastrophe for laptop battery life, among other resource use problems. And an LLM that fits in under 8 to 10GB of RAM for CPU-only inference is not going to be nearly as capable as an off-device inference system.

I wish they had just done this with a very clear up front opt in (not enabled by default) thing that explains what Mistral is, that it's not some big American cloud company but a relatively small startup in France, and that your prompts/LLM interactions will go to their servers. And some documentation on how it will be handled/stored in a supposedly trustworthy manner.

> This is an excellent use case for completely local, small model inference

Is it really ? LLM take lot of ram and drain battery. People run Firefox on low end computer.

The privacy policy for Smart Window[1] indicates that prompts are not only sent to an LLM provider (Mistral in this case), but also to Mozilla servers first.

  > The assistant sends the full prompt (including your query, any relevant Memories, and any additional relevant browsing context) to a Mozilla server.
  > 
  > Upon receipt, Mozilla forwards the request to a third-party large language model (LLM) on your behalf. The LLM receives the request from Mozilla, not directly from you, and sees a Mozilla IP address rather than your own. The request includes only the information needed to generate a response or perform the requested action.

  > When you submit a query in the Smart Bar, the assistant uses a local (on-device) intent classification model to determine whether the query is best addressed by a chat or a search.

I can't fathom how they'd consider "Private" something that uploads all the sensitive information in readable form to a third party service.

In fact, If I were asked to explain the opposite of private, I might give this kind of design as an example.

> This is an excellent use case for completely local, small model inference,

How small is this 'small model' inference?

Browsers especially are expected to operate on a very wide number of devices. It makes perfect sense to me to start off with a cloud driven approach and later allow people to configure it to run locally if they have the hardware to do so. Anyone with the hardware to do so is already signaling that they'd have the technical interest to make that change while having the default 'works for everyone' mode be one that doesn't require a powerful local device.

I say this as someone using a 6 year old computer and only upgraded to 8GB of ram when the latest WoW expansion was released.

Thanks for mentioning the mistake on the link, it's been reported internally.

its up to us to make extensions that are so famous that they can be integrated into the browser functionality i guess

Except ... this is not local. It's a remote service. Here is something that is:

    docker run ghcr.io/open-webui/open-webui:v0.11.3
    docker run mcr.microsoft.com/playwright/mcp:latest