Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Calibrated explanations: With uncertainty information and counterfactuals
Jönköping University, Jönköping International Business School, JIBS, Informatics.ORCID iD: 0000-0001-9633-0423
Jönköping University, School of Engineering, JTH, Department of Computing, Jönköping AI Lab (JAIL).ORCID iD: 0000-0003-0274-9026
Jönköping University, School of Engineering, JTH, Department of Computing, Jönköping AI Lab (JAIL).ORCID iD: 0000-0003-0412-6199
Jönköping University, School of Engineering, JTH, Department of Computing.ORCID iD: 0009-0009-0404-2586
2024 (English)In: Expert systems with applications, ISSN 0957-4174, E-ISSN 1873-6793, Vol. 246, article id 123154Article in journal (Refereed) Published
Abstract [en]

While local explanations for AI models can offer insights into individual predictions, such as feature importance, they are plagued by issues like instability. The unreliability of feature weights, often skewed due to poorly calibrated ML models, deepens these challenges. Moreover, the critical aspect of feature importance uncertainty remains mostly unaddressed in Explainable AI (XAI). The novel feature importance explanation method presented in this paper, called Calibrated Explanations (CE), is designed to tackle these issues head-on. Built on the foundation of Venn-Abers, CE not only calibrates the underlying model but also delivers reliable feature importance explanations with an exact definition of the feature weights. CE goes beyond conventional solutions by addressing output uncertainty. It accomplishes this by providing uncertainty quantification for both feature weights and the model’s probability estimates. Additionally, CE is model-agnostic, featuring easily comprehensible conditional rules and the ability to generate counterfactual explanations with embedded uncertainty quantification. Results from an evaluation with 25 benchmark datasets underscore the efficacy of CE, making it stand as a fast, reliable, stable, and robust solution.

Place, publisher, year, edition, pages
Elsevier, 2024. Vol. 246, article id 123154
Keywords [en]
Explainable AI, Feature Importance, Calibrated Explanations, Venn-Abers, Uncertainty Quantification, Counterfactual Explanations
National Category
Information Systems
Identifiers
URN: urn:nbn:se:hj:diva-62864DOI: 10.1016/j.eswa.2024.123154ISI: 001164089000001Scopus ID: 2-s2.0-85182588063Local ID: HOA;;1810433OAI: oai:DiVA.org:hj-62864DiVA, id: diva2:1810433
Funder
Knowledge Foundation, 20160035
Note

Included in doctoral thesis in manuscript form.

Available from: 2023-11-08 Created: 2023-11-08 Last updated: 2024-03-01Bibliographically approved
In thesis
1. Trustworthy explanations: Improved decision support through well-calibrated uncertainty quantification
Open this publication in new window or tab >>Trustworthy explanations: Improved decision support through well-calibrated uncertainty quantification
2023 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

The use of Artificial Intelligence (AI) has transformed fields like disease diagnosis and defence. Utilising sophisticated Machine Learning (ML) models, AI predicts future events based on historical data, introducing complexity that challenges understanding and decision-making. Previous research emphasizes users’ difficulty discerning when to trust predictions due to model complexity, underscoring addressing model complexity and providing transparent explanations as pivotal for facilitating high-quality decisions.

Many ML models offer probability estimates for predictions, commonly used in methods providing explanations to guide users on prediction confidence. However, these probabilities often do not accurately reflect the actual distribution in the data, leading to potential user misinterpretation of prediction trustworthiness. Additionally, most explanation methods fail to convey whether the model’s probability is linked to any uncertainty, further diminishing the reliability of the explanations.

Evaluating the quality of explanations for decision support is challenging, and although highlighted as essential in research, there are no benchmark criteria for comparative evaluations.

This thesis introduces an innovative explanation method that generates reliable explanations, incorporating uncertainty information supporting users in determining when to trust the model’s predictions. The thesis also outlines strategies for evaluating explanation quality and facilitating comparative evaluations. Through empirical evaluations and user studies, the thesis provides practical insights to support decision-making utilising complex ML models.

Abstract [sv]

Användningen av Artificiell intelligens (AI) har förändrat områden som diagnosticering av sjukdomar och försvar. Genom att använda sofistikerade maskininlärningsmodeller predicerar AI framtida händelser baserat på historisk data. Modellernas komplexitet resulterar samtidigt i utmanande beslutsprocesser när orsakerna till prediktionerna är svårbegripliga. Tidigare forskning pekar på användares problem att avgöra prediktioners tillförlitlighet på grund av modellkomplexitet och belyser vikten av att tillhandahålla transparenta förklaringar för att underlätta högkvalitativa beslut.

Många maskininlärningsmodeller erbjuder sannolikhetsuppskattningar för prediktionerna, vilket vanligtvis används i metoder som ger förklaringar för att vägleda användare om prediktionernas tillförlitlighet. Dessa sannolikheter återspeglar dock ofta inte de faktiska fördelningarna i datat, vilket kan leda till att användare felaktigt tolkar prediktioner som tillförlitliga. Därutöver förmedlar de flesta förklaringsmetoder inte om prediktionernas sannolikheter är kopplade till någon osäkerhet, vilket minskar tillförlitligheten hos förklaringarna.

Att utvärdera kvaliteten på förklaringar för beslutsstöd är utmanande, och även om det har betonats som avgörande i forskning finns det inga benchmark-kriterier för jämförande utvärderingar.

Denna avhandling introducerar en innovativ förklaringsmetod som genererar tillförlitliga förklaringar vilka inkluderar osäkerhetsinformation för att stödja användare att avgöra när man kan lita på modellens prediktioner. Avhandlingen ger också förslag på strategier för att utvärdera kvaliteten på förklaringar och underlätta jämförande utvärderingar. Genom empiriska utvärderingar och användarstudier ger avhandlingen praktiska insikter för att stödja beslutsfattande användande komplexa maskininlärningsmodeller.

Place, publisher, year, edition, pages
Jönköping: Jönköping University, Jönköping International Business School, 2023. p. 72
Series
JIBS Dissertation Series, ISSN 1403-0470 ; 159
Keywords
Explainable Artificial Intelligence, Interpretable Machine Learning, Decision Support Systems, Uncertainty Estimation, Explanation Methods
National Category
Information Systems, Social aspects Computer Sciences
Identifiers
urn:nbn:se:hj:diva-62865 (URN)978-91-7914-031-1 (ISBN)978-91-7914-032-8 (ISBN)
Public defence
2023-12-12, B1014, Jönköping International Business School, Jönköping, 13:15 (English)
Opponent
Supervisors
Available from: 2023-11-08 Created: 2023-11-08 Last updated: 2023-11-08Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopusPreprint (arXiv)

Authority records

Löfström, HelenaLöfström, TuweJohansson, UlfSönströd, Cecilia

Search in DiVA

By author/editor
Löfström, HelenaLöfström, TuweJohansson, UlfSönströd, Cecilia
By organisation
JIBS, InformaticsJönköping AI Lab (JAIL)JTH, Department of Computing
In the same journal
Expert systems with applications
Information Systems

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 293 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf