alt Hacker NewsEverything in AI seems to be focused on LLMs lately. But in my opinion, powerful problem-specific models like this are even more interesting. The SOTA AI models used in weather forecasting are already outperforming the classic NWP models while being orders of magnitude more efficient (inference). Most are based on multi scale (hierarchical) Graph Neural Networks, an architecture which is not often talked about. The original Graphcast paper is worth a read if you think this is interesting: https://arxiv.org/abs/2212.12794
Replies
You say this as if you don't need he MWP models to train the AI models? The accuracy of the AI Prediction depends entirely on the quality of the training dataset...
Rumor is that part of the disruption at GDM these past few months also involved people not wanting to be bound to strictly LLM research.
everything in AI is not focused on LLM, if you think so then that's because you are in LLM bubble. The big idea with LLM is that it's generative AI, the generative could be anything! Not just large languages, we have seen break through in image generation, video, audio, but guess what. Anything that you have enough data and given data you can predict what comes next can have gen AI applied, so we are seeing it with physical actions so robots get trained to generate the next move, and I think the same thing applies to weather forecast. It's predictable too given enough data
I can assure you that anyone who touches numerical simulations of any kind (physicists, engineers, chemists, biophysicts...etc) has tried their hand at ML based surrogate models in the last 5 years, so it's not like they aren't being tested. From my experience, they aren't very robust. Weather modeling is actually one of the very few areas where it seems to work half decently.
Insightful paper, thanks for sharing. Two things stand out to me.
First, it reinforces that you want methods that get better with more data. It emphasizes that the current approach cannot improve based on historic data - that’s the opportunity that ML based approaches exploit.
Second, it highlights that mature legacy solutions are tough competitors. They benefit from extensive tuning and real world feedback. Even when you have a genuinely better approach, it will take meaningful time & effort to achieve the current standard.
Patience and solid long term strategy are needed to make progress in these situations. You need confidence that your approach will win long term, backed by enough money & time to prove yourself correct.
One of my professors is referenced in the Wikipedia page of graph neural networks. It's funny that he explained them in the worse way possible and I eventually understood them better with another professor