Yeah, there have been several papers with attacks on Whisper:

- Inject adversarial noise to make it transcribe what you want (https://arxiv.org/abs/2210.17316)

- Stop it from transcribing (https://arxiv.org/abs/2405.06134)

- Adversarial prompt injection to make it translate instead of transcribe (https://arxiv.org/abs/2407.04482v2).