Change search
Link to record
Permanent link

Direct link
Publications (10 of 94) Show all publications
Löfström, H., Löfström, T., Johansson, U. & Sönströd, C. (2024). Calibrated explanations with uncertainty information and counterfactuals. Expert systems with applications, 246, Article ID 123154.
Open this publication in new window or tab >>Calibrated explanations with uncertainty information and counterfactuals
2024 (English)In: Expert systems with applications, ISSN 0957-4174, E-ISSN 1873-6793, Vol. 246, article id 123154Article in journal (Refereed) Published
Abstract [en]

While local explanations for AI models can offer insights into individual predictions, such as feature importance, they are plagued by issues like instability. The unreliability of feature weights, often skewed due to poorly calibrated ML models, deepens these challenges. Moreover, the critical aspect of feature importance uncertainty remains mostly unaddressed in Explainable AI (XAI). The novel feature importance explanation method presented in this paper, called Calibrated Explanations (CE), is designed to tackle these issues head-on. Built on the foundation of Venn-Abers, CE not only calibrates the underlying model but also delivers reliable feature importance explanations with an exact definition of the feature weights. CE goes beyond conventional solutions by addressing output uncertainty. It accomplishes this by providing uncertainty quantification for both feature weights and the model’s probability estimates. Additionally, CE is model-agnostic, featuring easily comprehensible conditional rules and the ability to generate counterfactual explanations with embedded uncertainty quantification. Results from an evaluation with 25 benchmark datasets underscore the efficacy of CE, making it stand as a fast, reliable, stable, and robust solution.

Place, publisher, year, edition, pages
Elsevier, 2024
Keywords
Explainable AI, Feature Importance, Calibrated Explanations, Venn-Abers, Uncertainty Quantification, Counterfactual Explanations
National Category
Information Systems
Identifiers
urn:nbn:se:hj:diva-62864 (URN)10.1016/j.eswa.2024.123154 (DOI)2-s2.0-85182588063 (Scopus ID)HOA;;1810433 (Local ID)HOA;;1810433 (Archive number)HOA;;1810433 (OAI)
Funder
Knowledge Foundation, 20160035
Note

Included in doctoral thesis in manuscript form.

Available from: 2023-11-08 Created: 2023-11-08 Last updated: 2024-01-30Bibliographically approved
Alkhatib, A., Boström, H., Ennadir, S. & Johansson, U. (2023). Approximating Score-based Explanation Techniques Using Conformal Regression. In: H. Papadopoulos, K. A. Nguyen, H. Boström, L. Carlsson (Ed.), Proceedings of Machine Learning Research: . Paper presented at 12th Symposium on Conformal and Probabilistic Prediction with Applications, COPA 2023 Limassol 13 September 2023 through 15 September 2023 (pp. 450-469). ML Research Press, 204
Open this publication in new window or tab >>Approximating Score-based Explanation Techniques Using Conformal Regression
2023 (English)In: Proceedings of Machine Learning Research / [ed] H. Papadopoulos, K. A. Nguyen, H. Boström, L. Carlsson, ML Research Press , 2023, Vol. 204, p. 450-469Conference paper, Published paper (Refereed)
Abstract [en]

Score-based explainable machine-learning techniques are often used to understand the logic behind black-box models. However, such explanation techniques are often computationally expensive, which limits their application in time-critical contexts. Therefore, we propose and investigate the use of computationally less costly regression models for approximating the output of score-based explanation techniques, such as SHAP. Moreover, validity guarantees for the approximated values are provided by the employed inductive conformal prediction framework. We propose several non-conformity measures designed to take the difficulty of approximating the explanations into account while keeping the computational cost low. We present results from a large-scale empirical investigation, in which the approximate explanations generated by our proposed models are evaluated with respect to efficiency (interval size). The results indicate that the proposed method can significantly improve execution time compared to the fast version of SHAP, TreeSHAP. The results also suggest that the proposed method can produce tight intervals, while providing validity guarantees. Moreover, the proposed approach allows for comparing explanations of different approximation methods and selecting a method based on how informative (tight) are the predicted intervals.

Place, publisher, year, edition, pages
ML Research Press, 2023
Keywords
Explainable machine learning, Inductive conformal prediction, Multi-target regression, Computation theory, Conformal mapping, Regression analysis, Black box modelling, Conformal predictions, Machine learning techniques, Machine-learning, Multi-targets, Target regression, Time-critical, Machine learning
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:hj:diva-63086 (URN)2-s2.0-85178664754 (Scopus ID)
Conference
12th Symposium on Conformal and Probabilistic Prediction with Applications, COPA 2023 Limassol 13 September 2023 through 15 September 2023
Funder
Knut and Alice Wallenberg Foundation
Available from: 2023-12-19 Created: 2023-12-19 Last updated: 2023-12-19Bibliographically approved
Johansson, U., Sönströd, C., Löfström, T. & Boström, H. (2023). Confidence Classifiers with Guaranteed Accuracy or Precision. In: H. Papadopoulos, K. A. Nguyen, H. Boström & L. Carlsson (Ed.), Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications: . Paper presented at Twelfth Symposium on Conformal and Probabilistic Prediction with Applications, 13-15 September 2023, Limassol, Cyprus (pp. 513-533). Proceedings of Machine Learning Research (PMLR), 204
Open this publication in new window or tab >>Confidence Classifiers with Guaranteed Accuracy or Precision
2023 (English)In: Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications / [ed] H. Papadopoulos, K. A. Nguyen, H. Boström & L. Carlsson, Proceedings of Machine Learning Research (PMLR) , 2023, Vol. 204, p. 513-533Conference paper, Published paper (Refereed)
Abstract [en]

In many situations, probabilistic predictors have replaced conformal classifiers. The main reason is arguably that the set predictions of conformal classifiers, with the accompanying significance level, are hard to interpret. In this paper, we demonstrate how conformal classification can be used as a basis for a classifier with reject option. Specifically, we introduce and evaluate two algorithms that are able to perfectly estimate accuracy or precision for a set of test instances, in a classifier with reject scenario. In the empirical investigation, the suggested algorithms are shown to clearly outperform both calibrated and uncalibrated probabilistic predictors.

Place, publisher, year, edition, pages
Proceedings of Machine Learning Research (PMLR), 2023
Series
Proceedings of Machine Learning Research, E-ISSN 2640-3498 ; 204
Keywords
Conformal prediction, Classification, Classification with reject option, Precision
National Category
Computer Sciences Information Systems
Identifiers
urn:nbn:se:hj:diva-62787 (URN)2-s2.0-85178665732 (Scopus ID)
Conference
Twelfth Symposium on Conformal and Probabilistic Prediction with Applications, 13-15 September 2023, Limassol, Cyprus
Funder
Knowledge Foundation
Available from: 2023-10-27 Created: 2023-10-27 Last updated: 2023-12-19Bibliographically approved
Johansson, U., Löfström, T., Sönströd, C. & Löfström, H. (2023). Conformal Prediction for Accuracy Guarantees in Classification with Reject Option. In: V. Torra and Y. Narukawa (Ed.), Modeling Decisions for Artificial Intelligence: 20th International Conference, MDAI 2023, Umeå, Sweden, June 19–22, 2023, Proceedings. Paper presented at International Conference on Modeling Decisions for Artificial Intelligence Umeå, Sweden 19 June 2023 (pp. 133-145). Springer
Open this publication in new window or tab >>Conformal Prediction for Accuracy Guarantees in Classification with Reject Option
2023 (English)In: Modeling Decisions for Artificial Intelligence: 20th International Conference, MDAI 2023, Umeå, Sweden, June 19–22, 2023, Proceedings / [ed] V. Torra and Y. Narukawa, Springer, 2023, p. 133-145Conference paper, Published paper (Refereed)
Abstract [en]

A standard classifier is forced to predict the label of every test instance, even when confidence in the predictions is very low. In many scenarios, it would, however, be better to avoid making these predictions, maybe leaving them to a human expert. A classifier with that alternative is referred to as a classifier with reject option. In this paper, we propose an algorithm that, for a particular data set, automatically suggests a number of accuracy levels, which it will be able to meet perfectly, using a classifier with reject option. Since the basis of the suggested algorithm is conformal prediction, it comes with strong validity guarantees. The experimentation, using 25 publicly available two-class data sets, confirms that the algorithm obtains empirical accuracies very close to the requested levels. In addition, in an outright comparison with probabilistic predictors, including models calibrated with Platt scaling, the suggested algorithm clearly outperforms the alternatives.

Place, publisher, year, edition, pages
Springer, 2023
Series
Lecture Notes in Computer Science, ISSN 2366-6323, E-ISSN 2366-6331 ; 13890
Keywords
Classification (of information), Accuracy level, Conformal predictions, Data set, Human expert, Probabilistics, Scalings, Test instances, Forecasting
National Category
Information Systems
Identifiers
urn:nbn:se:hj:diva-61450 (URN)10.1007/978-3-031-33498-6_9 (DOI)2-s2.0-85161105564 (Scopus ID)978-3-031-33497-9 (ISBN)
Conference
International Conference on Modeling Decisions for Artificial Intelligence Umeå, Sweden 19 June 2023
Available from: 2023-06-21 Created: 2023-06-21 Last updated: 2024-02-09Bibliographically approved
Johansson, U., Löfström, T. & Boström, H. (2023). Conformal Predictive Distribution Trees. Annals of Mathematics and Artificial Intelligence
Open this publication in new window or tab >>Conformal Predictive Distribution Trees
2023 (English)In: Annals of Mathematics and Artificial Intelligence, ISSN 1012-2443, E-ISSN 1573-7470Article in journal (Refereed) Epub ahead of print
Abstract [en]

Being able to understand the logic behind predictions or recommendations on the instance level is at the heart of trustworthy machine learning models. Inherently interpretable models make this possible by allowing inspection and analysis of the model itself, thus exhibiting the logic behind each prediction, while providing an opportunity to gain insights about the underlying domain. Another important criterion for trustworthiness is the model’s ability to somehow communicate a measure of confidence in every specific prediction or recommendation. Indeed, the overall goal of this paper is to produce highly informative models that combine interpretability and algorithmic confidence. For this purpose, we introduce conformal predictive distribution trees, which is a novel form of regression trees where each leaf contains a conformal predictive distribution. Using this representation language, the proposed approach allows very versatile analyses of individual leaves in the regression trees. Specifically, depending on the chosen level of detail, the leaves, in addition to the normal point predictions, can provide either cumulative distributions or prediction intervals that are guaranteed to be well-calibrated. In the empirical evaluation, the suggested conformal predictive distribution trees are compared to the well-established conformal regressors, thus demonstrating the benefits of the enhanced representation.

Place, publisher, year, edition, pages
Springer, 2023
Keywords
Conformal predictive distributions, Conformal regression, Interpretability, Regression trees
National Category
Computer Sciences
Identifiers
urn:nbn:se:hj:diva-61037 (URN)10.1007/s10472-023-09847-0 (DOI)000999966600001 ()2-s2.0-85160848450 (Scopus ID)HOA;;884987 (Local ID)HOA;;884987 (Archive number)HOA;;884987 (OAI)
Funder
Knowledge Foundation, 20200223
Available from: 2023-06-12 Created: 2023-06-12 Last updated: 2023-06-16
Löfström, H., Löfström, T., Johansson, U. & Sönströd, C. (2023). Investigating the impact of calibration on the quality of explanations. Annals of Mathematics and Artificial Intelligence
Open this publication in new window or tab >>Investigating the impact of calibration on the quality of explanations
2023 (English)In: Annals of Mathematics and Artificial Intelligence, ISSN 1012-2443, E-ISSN 1573-7470Article in journal (Refereed) Epub ahead of print
Abstract [en]

Predictive models used in Decision Support Systems (DSS) are often requested to explain the reasoning to users. Explanations of instances consist of two parts; the predicted label with an associated certainty and a set of weights, one per feature, describing how each feature contributes to the prediction for the particular instance. In techniques like Local Interpretable Model-agnostic Explanations (LIME), the probability estimate from the underlying model is used as a measurement of certainty; consequently, the feature weights represent how each feature contributes to the probability estimate. It is, however, well-known that probability estimates from classifiers are often poorly calibrated, i.e., the probability estimates do not correspond to the actual probabilities of being correct. With this in mind, explanations from techniques like LIME risk becoming misleading since the feature weights will only describe how each feature contributes to the possibly inaccurate probability estimate. This paper investigates the impact of calibrating predictive models before applying LIME. The study includes 25 benchmark data sets, using Random forest and Extreme Gradient Boosting (xGBoost) as learners and Venn-Abers and Platt scaling as calibration methods. Results from the study show that explanations of better calibrated models are themselves better calibrated, with ECE and log loss for the explanations after calibration becoming more conformed to the model ECE and log loss. The conclusion is that calibration makes the models and the explanations better by accurately representing reality.

Place, publisher, year, edition, pages
Springer, 2023
Keywords
Calibration, Decision support systems, Explainable artificial intelligence, Predicting with confidence, Uncertainty in explanations, Venn Abers
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:hj:diva-60033 (URN)10.1007/s10472-023-09837-2 (DOI)000948763400001 ()2-s2.0-85149810932 (Scopus ID)HOA;;870772 (Local ID)HOA;;870772 (Archive number)HOA;;870772 (OAI)
Funder
Knowledge Foundation
Available from: 2023-03-27 Created: 2023-03-27 Last updated: 2023-11-08
Löfström, T., Bondaletov, A., Ryasik, A., Boström, H. & Johansson, U. (2023). Tutorial on using Conformal Predictive Systems in KNIME. In: H. Papadopoulos, K. A. Nguyen, H. Boström & L. Carlsson (Ed.), Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications: . Paper presented at Twelfth Symposium on Conformal and Probabilistic Prediction with Applications, 13-15 September 2023, Limassol, Cyprus (pp. 602-620). Proceedings of Machine Learning Research (PMLR), 204
Open this publication in new window or tab >>Tutorial on using Conformal Predictive Systems in KNIME
Show others...
2023 (English)In: Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications / [ed] H. Papadopoulos, K. A. Nguyen, H. Boström & L. Carlsson, Proceedings of Machine Learning Research (PMLR) , 2023, Vol. 204, p. 602-620Conference paper, Published paper (Refereed)
Abstract [en]

KNIME is an end-to-end software platform for data science with an open-source analytics platform for creating solutions and a commercial server solution for productionization. Conformal classification and regression have previously been implemented in KNIME. We extend the conformal prediction package with added support for conformal predictive systems, taking inspiration from the interface of the Crepes package in Python. The paper demonstrates some typical use cases for conformal predictive systems. Furthermore, the paper also illustrates how to create Mondrian conformal predictors using the KNIME implementation. All examples are publicly available, and the package is1 available through KNIME's official software repositories.

Place, publisher, year, edition, pages
Proceedings of Machine Learning Research (PMLR), 2023
Series
Proceedings of Machine Learning Research, E-ISSN 2640-3498 ; 204
National Category
Computer Sciences Information Systems
Identifiers
urn:nbn:se:hj:diva-62788 (URN)2-s2.0-85178664607 (Scopus ID)
Conference
Twelfth Symposium on Conformal and Probabilistic Prediction with Applications, 13-15 September 2023, Limassol, Cyprus
Funder
Knowledge Foundation
Available from: 2023-10-27 Created: 2023-10-27 Last updated: 2023-12-19Bibliographically approved
Löfström, H., Hammar, K. & Johansson, U. (2022). A meta survey of quality evaluation criteria in explanation methods. In: J. De Weerdt, Jochen & A. Polyvyanyy (Ed.), Intelligent Information Systems: CAiSE Forum 2022, Leuven, Belgium, June 6–10, 2022, Proceedings. Paper presented at CAiSE Forum 2022, Leuven, Belgium, June 6–10, 2022 (pp. 55-63). Cham: Springer
Open this publication in new window or tab >>A meta survey of quality evaluation criteria in explanation methods
2022 (English)In: Intelligent Information Systems: CAiSE Forum 2022, Leuven, Belgium, June 6–10, 2022, Proceedings / [ed] J. De Weerdt, Jochen & A. Polyvyanyy, Cham: Springer, 2022, p. 55-63Conference paper, Published paper (Refereed)
Abstract [en]

The evaluation of explanation methods has become a significant issue in explainable artificial intelligence (XAI) due to the recent surge of opaque AI models in decision support systems (DSS). Explanations are essential for bias detection and control of uncertainty since most accurate AI models are opaque with low transparency and comprehensibility. There are numerous criteria to choose from when evaluating explanation method quality. However, since existing criteria focus on evaluating single explanation methods, it is not obvious how to compare the quality of different methods.

Place, publisher, year, edition, pages
Cham: Springer, 2022
Series
Lecture Notes in Business Information Processing, ISSN 1865-1348, E-ISSN 1865-1356 ; 452
Keywords
Explanation method, Evaluation metric, Explainable artificial intelligence, Evaluation of explainability, Comparative evaluations
National Category
Computer Sciences
Identifiers
urn:nbn:se:hj:diva-57114 (URN)10.1007/978-3-031-07481-3_7 (DOI)978-3-031-07480-6 (ISBN)978-3-031-07481-3 (ISBN)
Conference
CAiSE Forum 2022, Leuven, Belgium, June 6–10, 2022
Funder
Knowledge Foundation
Available from: 2022-06-13 Created: 2022-06-13 Last updated: 2023-11-08Bibliographically approved
Alkhatib, A., Boström, H. & Johansson, U. (2022). Assessing Explanation Quality by Venn Prediction. In: U. Johansson, H. Boström, K. A. Nguyen, Z. Luo & L. Carlsson (Ed.), Proceedings of the Eleventh Symposium on Conformal and Probabilistic Prediction with Applications: Volume 179: Conformal and Probabilistic Prediction with Applications, 24-26 August 2022, Brighton, UK. Paper presented at 11th Symposium on Conformal and Probabilistic Prediction with Applications, 24-26 August 2022, Brighton, UK (pp. 42-54). ML Research Press, 179
Open this publication in new window or tab >>Assessing Explanation Quality by Venn Prediction
2022 (English)In: Proceedings of the Eleventh Symposium on Conformal and Probabilistic Prediction with Applications: Volume 179: Conformal and Probabilistic Prediction with Applications, 24-26 August 2022, Brighton, UK / [ed] U. Johansson, H. Boström, K. A. Nguyen, Z. Luo & L. Carlsson, ML Research Press , 2022, Vol. 179, p. 42-54Conference paper, Published paper (Refereed)
Abstract [en]

Rules output by explainable machine learning techniques naturally come with a degree of uncertainty, as the complex functionality of the underlying black-box model often can be difficult to approximate by a single, interpretable rule. However, the uncertainty of these approximations is not properly quantified by current explanatory techniques. The use of Venn prediction is here proposed and investigated as a means to quantify the uncertainty of the explanations and thereby also allow for competing explanation techniques to be evaluated with respect to their relative uncertainty. A number of metrics of rule explanation quality based on uncertainty are proposed and discussed, including metrics that capture the tendency of the explanations to predict the correct outcome of a black-box model on new instances, how informative (tight) the produced intervals are, and how certain a rule is when predicting one class. An empirical investigation is presented, in which explanations produced by the state-of-the-art technique Anchors are compared to explanatory rules obtained from association rule mining. The results suggest that the association rule mining approach may provide explanations with less uncertainty towards the correct label, as predicted by the black-box model, compared to Anchors. The results also show that the explanatory rules obtained through association rule mining result in tighter intervals and are closer to either one or zero compared to Anchors, i.e., they are more certain towards a specific class label.

Place, publisher, year, edition, pages
ML Research Press, 2022
Series
Proceedings of Machine Learning Research, E-ISSN 2640-3498 ; 179
Keywords
Venn prediction, Explainable machine learning, Rule mining
National Category
Computer Sciences
Identifiers
urn:nbn:se:hj:diva-58686 (URN)2-s2.0-85164705716 (Scopus ID)
Conference
11th Symposium on Conformal and Probabilistic Prediction with Applications, 24-26 August 2022, Brighton, UK
Available from: 2022-10-24 Created: 2022-10-24 Last updated: 2023-08-17Bibliographically approved
Sweidan, D., Johansson, U., Gidenstam, A. & Alenljung, B. (2022). Predicting Customer Churn in Retailing. In: International Conference on Machine Learning and Applications (ICMLA): 21st IEEE International Conference on Machine Learning and Applications (ICMLA). Paper presented at Proceedings - 21st IEEE International Conference on Machine Learning and Applications, ICMLA 2022 Nassau 12 December 2022 through 14 December 2022 (pp. 635-640). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Predicting Customer Churn in Retailing
2022 (English)In: International Conference on Machine Learning and Applications (ICMLA): 21st IEEE International Conference on Machine Learning and Applications (ICMLA), Institute of Electrical and Electronics Engineers (IEEE), 2022, p. 635-640Conference paper, Published paper (Refereed)
Abstract [en]

Customer churn is one of the most challenging problems for digital retailers. With significantly higher costs for acquiring new customers than retaining existing ones, knowledge about which customers are likely to churn becomes essential. This paper reports a case study where a data-driven approach to churn prediction is used for predicting churners and gaining insights about the problem domain. The real-world data set used contains approximately 200 000 customers, describing each customer using more than 50 features. In the pre-processing, exploration, modeling and analysis, attributes related to recency, frequency, and monetary concepts are identified and utilized. In addition, correlations and feature importance are used to discover and understand churn indicators. One important finding is that the churn rate highly depends on the number of previous purchases. In the segment consisting of customers with only one previous purchase, more than 75% will churn, i.e., not make another purchase in the coming year. For customers with at least four previous purchases, the corresponding churn rate is around 25%. Further analysis shows that churning customers in general, and as expected, make smaller purchases and visit the online store less often. In the experimentation, three modeling techniques are evaluated, and the results show that, in particular, Gradient Boosting models can predict churners with relatively high accuracy while obtaining a good balance between precision and recall.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2022
Keywords
Sales, Case-studies, Churn rates, Correlation, Customer churn prediction, Customer churns, Digital retailing, Feature importance, High costs, RFM analysis, Top probability, Forecasting, correlations, top probabilities
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:hj:diva-60233 (URN)10.1109/ICMLA55696.2022.00105 (DOI)2-s2.0-85152214345 (Scopus ID)978-1-6654-6283-9 (ISBN)
Conference
Proceedings - 21st IEEE International Conference on Machine Learning and Applications, ICMLA 2022 Nassau 12 December 2022 through 14 December 2022
Funder
Knowledge Foundation, 20160035, 20170215
Available from: 2023-04-24 Created: 2023-04-24 Last updated: 2023-10-04Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0003-0412-6199

Search in DiVA

Show all publications