So to make this profitable they need ads revenue from it, right? Imagine for a moment the ways AI can manipulate responses and conversations for marketers, because I guarantee the marketers have already thought about it.
The fact that steering one of these things is trivial nowadays and the vectors are close-to-free-to-store (since you don't need anything large to influence the space, see also https://www.youtube.com/watch?v=ahtbcExEKng) means that this is very likely already happening.
Anthropic was talking about this as a "oh nifty, look at this" back in 2024: https://www.anthropic.com/news/golden-gate-claude
The fact that steering one of these things is trivial nowadays and the vectors are close-to-free-to-store (since you don't need anything large to influence the space, see also https://www.youtube.com/watch?v=ahtbcExEKng) means that this is very likely already happening.