Planned maintenance
A system upgrade is planned for 10/12-2024, at 12:00-13:00. During this time DiVA will be unavailable.
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Investigating Normalized Conformal Regressors
Jönköping University, School of Engineering, JTH, Department of Computing, Jönköping AI Lab (JAIL).ORCID iD: 0000-0003-0412-6199
School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Sweden.
Jönköping University, School of Engineering, JTH, Department of Computing, Jönköping AI Lab (JAIL).ORCID iD: 0000-0003-0274-9026
2021 (English)In: 2021 IEEE Symposium Series on Computational Intelligence, SSCI 2021 - Proceedings, Institute of Electrical and Electronics Engineers (IEEE), 2021Conference paper, Published paper (Other academic)
Abstract [en]

Conformal prediction can be applied on top of any machine learning predictive regression model, thus turning it into a conformal regressor. Given a significance level $\epsilon$, conformal regressors output valid prediction intervals, i.e., the probability that the interval covers the true value is exactly $1-\epsilon$. To obtain validity, a calibration set that is not used for training the model must be set aside. In standard inductive conformal regression, the size of the prediction intervals is then determined by the absolute error made by the predictive model on a specific instance in the calibration set, where different significance levels correspond to different instances. In this setting, all prediction intervals will have the same size, making the resulting models very unspecific. When adding a technique called normalization, however, the difficulty of each instance is estimated, and the interval sizes are adjusted accordingly. An integral part of normalized conformal regressors is a parameter called $\beta$, which determines the relative importance of the difficulty estimation and the error of the model. In this study, the effects of different underlying models, difficulty estimation functions and $\beta$ -values are investigated. The results from a large empirical study, using twenty publicly available data sets, show that better difficulty estimation functions will lead to both tighter and more specific prediction intervals. Furthermore, it is found that the $\beta$ -values used strongly affect the conformal regressor. While there is no specific $\beta$ -value that will always minimize the interval sizes, lower $\beta$ -values lead to more variation in the interval sizes, i.e., more specific models. In addition, the analysis also identifies that the normalization procedure introduces a small but unfortunate bias in the models. More specifically, normalization using low $\beta$ -values means that smaller intervals are more likely to be erroneous, while the opposite is true for higher $\beta$ -values. © 2021 IEEE.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2021.
Keywords [en]
Conformal prediction, Gradient boosting, Predictive regression, Random forest, Conformal mapping, Decision trees, Forecasting, Regression analysis, Conformal predictions, Difficulty estimations, Estimation function, Interval size, Normalisation, Prediction interval, Random forests, Significance levels, Calibration
National Category
Computer Sciences
Identifiers
URN: urn:nbn:se:hj:diva-56037DOI: 10.1109/SSCI50451.2021.9659853Scopus ID: 2-s2.0-85125760858ISBN: 9781728190488 (print)OAI: oai:DiVA.org:hj-56037DiVA, id: diva2:1644705
Conference
2021 IEEE Symposium Series on Computational Intelligence, SSCI 2021, 5 December 2021 through 7 December 2021
Available from: 2022-03-15 Created: 2022-03-15 Last updated: 2022-03-15Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Johansson, UlfLöfström, Tuwe

Search in DiVA

By author/editor
Johansson, UlfLöfström, Tuwe
By organisation
Jönköping AI Lab (JAIL)
Computer Sciences

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 85 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf