Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Conformal Prediction for Accuracy Guarantees in Classification with Reject Option
Jönköping University, School of Engineering, JTH, Department of Computing, Jönköping AI Lab (JAIL).ORCID iD: 0000-0003-0412-6199
Jönköping University, School of Engineering, JTH, Department of Computing, Jönköping AI Lab (JAIL).ORCID iD: 0000-0003-0274-9026
Jönköping University, School of Engineering, JTH, Department of Computing.ORCID iD: 0009-0009-0404-2586
Jönköping University, Jönköping International Business School.ORCID iD: 0000-0001-9633-0423
2023 (English)In: Modeling Decisions for Artificial Intelligence: 20th International Conference, MDAI 2023, Umeå, Sweden, June 19–22, 2023, Proceedings / [ed] V. Torra and Y. Narukawa, Springer, 2023, p. 133-145Conference paper, Published paper (Refereed)
Abstract [en]

A standard classifier is forced to predict the label of every test instance, even when confidence in the predictions is very low. In many scenarios, it would, however, be better to avoid making these predictions, maybe leaving them to a human expert. A classifier with that alternative is referred to as a classifier with reject option. In this paper, we propose an algorithm that, for a particular data set, automatically suggests a number of accuracy levels, which it will be able to meet perfectly, using a classifier with reject option. Since the basis of the suggested algorithm is conformal prediction, it comes with strong validity guarantees. The experimentation, using 25 publicly available two-class data sets, confirms that the algorithm obtains empirical accuracies very close to the requested levels. In addition, in an outright comparison with probabilistic predictors, including models calibrated with Platt scaling, the suggested algorithm clearly outperforms the alternatives.

Place, publisher, year, edition, pages
Springer, 2023. p. 133-145
Series
Lecture Notes in Computer Science, ISSN 2366-6323, E-ISSN 2366-6331 ; 13890
Keywords [en]
Classification (of information), Accuracy level, Conformal predictions, Data set, Human expert, Probabilistics, Scalings, Test instances, Forecasting
National Category
Information Systems
Identifiers
URN: urn:nbn:se:hj:diva-61450DOI: 10.1007/978-3-031-33498-6_9Scopus ID: 2-s2.0-85161105564ISBN: 978-3-031-33497-9 (print)OAI: oai:DiVA.org:hj-61450DiVA, id: diva2:1772471
Conference
International Conference on Modeling Decisions for Artificial Intelligence Umeå, Sweden 19 June 2023
Available from: 2023-06-21 Created: 2023-06-21 Last updated: 2024-02-09Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Johansson, UlfLöfström, TuweSönströd, CeciliaLöfström, Helena

Search in DiVA

By author/editor
Johansson, UlfLöfström, TuweSönströd, CeciliaLöfström, Helena
By organisation
Jönköping AI Lab (JAIL)JTH, Department of ComputingJönköping International Business School
Information Systems

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 115 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf