Machine learning _has_ been widely used in weather forecasting, but in a different way than these models. Going back to the 1970's, you never just take the output of a numerical weather model and call it a forecast. We know that limitations in the models' resolution and representation of physical processes lead to huge biases and missed details that cause the forecast to disagree with real world observations. So a standard technique has been to post-process model outputs, calibrating them for station observations where available. You don't need super complex ML to really dramatically improve the quality or skill of the forecast in this manner; typically multiple linear regressions with some degree of feature selection and other criteria will capture most of the variance, especially when you pool observation stations together.

> Are there any other benefits? Like is there a reason to believe it could be more accurate than a physics model with some error bars?

Surprisingly, the leading AI-NWP forecasts are more accurate than their traditional counterparts, even at large scales and long lead times (i.e. the 5-day forecast).

The reason for this is not at all obvious, to the point I'd call it an open question in the literature. Large-scale atmospheric dynamics are a well-studied domain, so physics-based models essentially have to be getting "the big stuff" right. It's reasonable to think that AI-NWP models are doing a better job at sub-grid parameterizations and local forcings because those are the 'gaps' in traditional NWP, but going from "improved modelling of turbulence over urban and forest areas" (as a hypothetical example) to "improvements in 10,000 km-scale atmospheric circulation 5 days later" isn't as certain.