IEA Wind Task 51 Workshop AI driven Weather Forecast models

On the 11th of January, I attended a webinar conference organized by IEA Wind Task 51 on Machine Learning Weather Prediction.

 

The talk has been recorded and is available on YouTube. The three-hour presentation was given by four researchers :

• Mariana Clare from ECMWF on "The rise of data driven weather forecasting"
• Gregory Hakim from the University of Washington on "Dynamical Test of a Deep-Learning Weather Prediction Model"
• Joel Oskarsson from Linköping University on "Neural Weather Prediction for Limited Area Modelling"
• Florian Achermann from ETH and JUA.ai on "Ultra High Resolution Wind
  Forecasts"

The first presentation from Mariana Clare gives an overview of the current state of the work done by ECMWF on data driven weather forecasting.

Since 2022, ten AI models have been developed by different companies and/or universities. Since then, the European Centre for Medium-Range Weather Forecasts have run deep analysis on a few chosen AI models:

• FourCastNet by NVIDIA (Early 2022)
• PanguWeather by Huawei (Late 2022)
• GraphCast by DeepMind (Late 2022)

The test is split in two categories. First, the skill of each model is compared with the classical IFS HRES model from ECMWF. The results for different variables are shown below.

A scoreboard represent the skill of a given forecast compared to the IFS HRES model (state of the art for physics-based models) (source : Blog)

FourCastNet is not shown here, but this image highlights how well PanguWeather, but mostly, GraphCast does compare to the ECMWF model.

Skill score are one thing when assessing the performance of weather models, but extreme events are even more important. The next part of the test is then to analyse certain past events and see how well AI model are able to depict them.

Dr Clare, proceeds on looking at two particular events : Storm Eunice, which hit Great Britain in 2022 and the Cyclone Freddy which occurred in the Indian Ocean. The different models (IFS HRES, FourCastNet, PanguWeather, and GraphCast) are all able to predict the storm 60 hours in the future. However, AI models (PanguWeather and FourCastNet) do not show the same results for the prediction of the Cyclone 48 hours in the future compared to the HRES model. 

The AI models assessed are all trained using 40 years of reanalysis data from ERA5 using multiple variables (For GraphCast the trained data set is approximately 35GB). Then, the models are fine-tuned using a more accurate analysis for specific time periods. As the reanalysis dataset, the models have a spatial resolution of 0.25°.

The main advantages of AI models are the time and energy consumption of one forecast. After being trained, AI models are 500,000 less time and energy consuming than a classic HRES forecast. ECMWF have developed python modules for each of the three AI models which can be installed on a local machine :

• pip install ai-models-panguweather
• pip install ai-models-fourcastnetv2
• pip install ai-models-graphcast

While analysing these different models, ECMWF also wanted to develop their own using a similar approach to the DeepMind model. The first version seems to show good results, but their main goal is to increase the spatial resolution.

 

She finishes the talk by mentioning the challenges in the years to come for AI models :

1. Train the models on observation as well the reanalysis.
2. Move from deterministic to probabilistic models to integrate measures of uncertainty.
3. Build trust in AI models for researcher.
   

The next talk by Gregory Hakim gives the detailed process of testing a deep learning weather model.

Joel Oskarsson then, gives an explanation of how AI models can run on a limited area.

Florian Achermann finishes by explaining the development of a high resolution wind forecasting model.

 

Boris