It'll be interesting to see if we get better / worse at figuring out what effective LLM interaction should look like, since that seems prerequisite to this kind of evaluation. Right now there's so little data on whether e.g. casual conversational interaction works better than structured prompting, and so, so, so much superstition - I don't know if I'd really trust judges to have good instincts about what's effective rather than what resembles their own use patterns.