I was using Grok with speech and discovered their "paralinguistic" information storage. At first it claimed that the storage was temporary, but then admitted that long term data was stored for training.

Some of the dimensions they store are prosody, intensity, timbre, non-verbal vocalizations, pauses, timing and emotional inflection. In other words, another large layer of information on top of just the prompt text. This data doesn't get translated into text, it goes straight into a speech-to-speech model.

It strikes me that from just a few minutes of such data and the associated semantic content, an AI can assemble a detailed and accurate emotional/psychological dossier of any user, on demand. In the hands of a federal agent it would be a powerful tool to impose their department's will, or their own. Also it's an ad targeting mother load. And if that were already in place we would have no way to know.

Talking to a machine seems banal already, but the metadata contains an instruction manual on where your buttons are and how to press them.