We had this exact same problem (the key initially wasn’t a secret but became a secret once we enabled Gemini API with no warnings).

We managed to catch it somewhat early through alerting, so the damage was only $26k.

We asked our Google cloud support rep for a refund - they initially came back with a no but now the case is under further consideration.

I’d escalate this up the chain as much as possible.