> The main goal is to make search better.

It is actually, but the shady part is that this is not (just) to target ads to you, though that certainly happens… it is, as OP explained, so that they can better capture query intent AND your behavior (clicks, dwell time, subsequent searches, etc) that they then use to refine and rerank search results, which improves Google Search and keeps you using it, which becomes a virtuous flywheel that competitors can’t catch up to.

One way for competitors to break through is to buy traffic, but well Google blocked that too by using its monopoly profits to pay more than anybody else (Apple and Mozilla search deals.)

All of this came out in the Search Antitrust trial. Look up NavBoost for instance.