Conceptos Básicos
Equipping deep neural networks with predictive uncertainty can improve the accuracy of tropical cyclone wind speed estimation from satellite data, especially for higher hurricane categories.
Resumen
The paper presents a theoretical and quantitative comparison of existing uncertainty quantification (UQ) methods for deep neural networks (DNNs) applied to the task of wind speed estimation in satellite imagery of tropical cyclones. The authors use a dataset of 25,000 infrared satellite images matched with storm data to train and evaluate various UQ methods, including deterministic, ensemble, Bayesian, quantile, and diffusion-based approaches.
The key findings are:
- Selective prediction enabled by uncertainty-aware models can yield significant accuracy improvements, with the best performing methods obtaining an RMSE between 9.27 - 10.95 knots.
- The effectiveness of selective prediction varies across different UQ methods and storm categories. Methods like SWAG and Conformalized Quantile Regression (CQR) achieve relatively low RMSE after selective prediction while maintaining reasonable coverage.
- The predictive uncertainty of the models correlates better with the error for lower storm categories (Tropical Depression) compared to higher hurricane categories. Certain methods like MVE, SWAG, and Quantile Regression demonstrate better calibration and sharpness of predictive uncertainties across categories.
- The authors provide a detailed discussion of the strengths and weaknesses of each UQ method group, highlighting their performance on metrics like RMSE, CRPS, NLL, and MACE.
The paper demonstrates the potential of uncertainty-aware deep learning models to enhance the quality of tropical cyclone wind speed estimation from satellite data, particularly for rapidly intensifying storms near coastlines.
Estadísticas
Tropical cyclones in the US alone have led to 6,789 deaths and caused $1,333.6 billion in financial damages between 1980-2022.
Hurricane Otis in October 2023 underwent a rapid intensification of almost 80 knots in 12 hours before causing devastating damage in Acapulco, Mexico.
The dataset used contains 53,000 training, 11,000 validation, and 43,000 test samples of infrared satellite imagery matched with storm data.
The distribution of wind speed targets is highly skewed, with the majority of samples falling below hurricane categories.
Citas
"Because data to train such prediction methods can be limited and unevenly distributed, making a perfect prediction is not always possible. However, based on the general viability of DNNs for predicting and estimating wind speeds from satellite data, one possible approach is to equip DNNs with modern uncertainty-quantification (UQ) methods to enhance the quality of predictions and mitigate data imbalances, as well as label and input noise."
"We find that predictive uncertainties can be utilized to further improve accuracy and analyze the predictive uncertainties of different methods across storm categories."