The inference time results are quite different. These models haven’t just been trained with tons of docs, blog entries and videos about Java 8. They’ve also been trained on tons of code examples from simple to complex real world ones.

You’re not going to get all that just by pointing the LLM to the recently released Java 27 documentation. That information is also potentially adding tons of content to your context, which is already filled with tons of other data (your code, other recently released libraries it has to get documentation for, etc).