This is kind of funny, but very much expected.

The interface into the LLM is tokens in and out (text, images, audio). And the harness generally doesn't understand what you're passing in. The LLM has nothing to do other than to respond with tokens and empty responses (eg. just a stop token) have been aggressively trained out of it.