Evaluation of conformal-based probabilistic forecasting methods for short-term wind speed forecasting
2023 (English)In: Proceedings of Machine Learning Research / [ed] H. Papadopoulos, K. A. Nguyen, Henrik Boström, L. Carlsson, ML Research Press , 2023, Vol. 204, p. 100-115Conference paper, Published paper (Refereed)
Abstract [en]
We apply Conformal Predictive Distribution Systems (CPDS) and a non-exchangeable version of the traditional Conformal Prediction (NECP) method to short-term wind speed forecasting to generate probabilistic forecasts. These are compared to the more traditional Quantile Regression Forest (QRF) method. A short-term forecast is available from a few hours before the forecasted time period and is only extended a couple days into the future. The methods are supplied ensemble forecasts as input and additionally the Conformal methods are supplied with post-processed point forecasts for generating the probability distributions. In the NECP case we propose a method of producing probability distributions by creating sequentially larger prediction intervals. The methods are compared through a teaching schedule, to mimic a real-world setting. For each model update in the teaching schedule a grid-search approach is applied to select each method’s optimal hyperparameters, respectively. The methods are tested out of the box with tweaks to few hyperparameters. We also introduce a normalized nonconformity score and use it with the conformal method that handles data that violates the exchangeability assumption. The resulting probability distributions are compared to actual wind measurements through Continuous Ranked Probability Scores (CRPS) as well as their validity and efficiency of certain prediction intervals. Our results suggest that the conformal based methods, with the pre-trained underlying model, produce slightly more conservative but more efficient probability distributions than QRF at a lower computational cost. We further propose how the conformal-based methods could be improved for the application to real-world scenarios.
Place, publisher, year, edition, pages
ML Research Press , 2023. Vol. 204, p. 100-115
Keywords [en]
conformal predictive distribution systems, exchangeable, non exchangeable, normalized nonconformity, quantile regression forests, short-term wind forecast, Probability distributions, Wind speed, Conformal predictive distribution system, Distribution systems, Predictive distributions, Quantile regression, Quantile regression forest, Regression forests, Short term wind forecast, Weather forecasting
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:hj:diva-63087Scopus ID: 2-s2.0-85178657296OAI: oai:DiVA.org:hj-63087DiVA, id: diva2:1821214
Conference
12th Symposium on Conformal and Probabilistic Prediction with Applications, COPA 2023 Limassol 13 September 2023 through 15 September 2023
2023-12-192023-12-192024-01-11Bibliographically approved