vente ... |

Link to record
http://hj.diva-portal.org/smash/person.jsf?pid=authority-person:53306 $(function(){PrimeFaces.cw("InputTextarea","widget_formSmash_upper_j_idt122_recordDirectLink",{id:"formSmash:upper:j_idt122:recordDirectLink",widgetVar:"widget_formSmash_upper_j_idt122_recordDirectLink",autoResize:true});}); $(function(){PrimeFaces.cw("OverlayPanel","widget_formSmash_upper_j_idt122_j_idt124",{id:"formSmash:upper:j_idt122:j_idt124",widgetVar:"widget_formSmash_upper_j_idt122_j_idt124",target:"formSmash:upper:j_idt122:permLink",showEffect:"blind",hideEffect:"fade",my:"right top",at:"right bottom",showCloseIcon:true});});

Permanent link

Direct link

Johansson, Ulforcid.org/0000-0003-0412-6199

Åpne denne publikasjonen i ny fane eller vindu >>Accurate Hit Estimation for Iterative Screening Using Venn-ABERS Predictors### Buendia, Ruben

### Kogej, Thierry

### Engkvist, Ola

### Carlsson, Lars

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_0_j_idt188_some",{id:"formSmash:j_idt184:0:j_idt188:some",widgetVar:"widget_formSmash_j_idt184_0_j_idt188_some",multiple:true}); ### Linusson, Henrik

### Johansson, Ulf

### Toccaceli, Paolo

### Ahlberg, Ernst

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_0_j_idt188_otherAuthors",{id:"formSmash:j_idt184:0:j_idt188:otherAuthors",widgetVar:"widget_formSmash_j_idt184_0_j_idt188_otherAuthors",multiple:true}); Vise andre…PrimeFaces.cw("SelectBooleanButton","widget_formSmash_j_idt184_0_j_idt188_j_idt202",{id:"formSmash:j_idt184:0:j_idt188:j_idt202",widgetVar:"widget_formSmash_j_idt184_0_j_idt188_j_idt202",onLabel:"Skjul andre\u2026",offLabel:"Vise andre\u2026"}); 2019 (engelsk)Inngår i: Journal of Chemical Information and Modeling, ISSN 1549-9596, E-ISSN 1549-960X, Vol. 59, nr 3, s. 1230-1237Artikkel i tidsskrift (Fagfellevurdert) Published
##### Abstract [en]

##### sted, utgiver, år, opplag, sider

American Chemical Society (ACS), 2019
##### HSV kategori

##### Identifikatorer

urn:nbn:se:hj:diva-43510 (URN)10.1021/acs.jcim.8b00724 (DOI)000462943700027 ()30726080 (PubMedID)2-s2.0-85063371683 (Scopus ID);JTHDatateknikIS (Lokal ID);JTHDatateknikIS (Arkivnummer);JTHDatateknikIS (OAI)
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_0_j_idt188_j_idt359",{id:"formSmash:j_idt184:0:j_idt188:j_idt359",widgetVar:"widget_formSmash_j_idt184_0_j_idt188_j_idt359",multiple:true});
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_0_j_idt188_j_idt365",{id:"formSmash:j_idt184:0:j_idt188:j_idt365",widgetVar:"widget_formSmash_j_idt184_0_j_idt188_j_idt365",multiple:true});
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_0_j_idt188_j_idt371",{id:"formSmash:j_idt184:0:j_idt188:j_idt371",widgetVar:"widget_formSmash_j_idt184_0_j_idt188_j_idt371",multiple:true});
#####

##### Forskningsfinansiär

Knowledge Foundation, 20150185
Tilgjengelig fra: 2019-04-23 Laget: 2019-04-23 Sist oppdatert: 2019-08-22bibliografisk kontrollert

Department of Information Technology, University of Borås, Borås, Sweden.

Discovery Sciences, AstraZeneca IMED Biotech Unit, Mölndal, Sweden.

Discovery Sciences, AstraZeneca IMED Biotech Unit, Mölndal, Sweden.

Discovery Sciences, AstraZeneca IMED Biotech Unit, Mölndal, Sweden.

Department of Information Technology, University of Borås, Borås, Sweden.

Högskolan i Jönköping, Tekniska Högskolan, JTH, Datateknik och informatik, JTH, Jönköping AI Lab (JAIL).

Department of Computer Science, Royal Holloway, University of London, Egham, Surrey, United Kingdom.

Data Science and AI, Drug Safety & Metabolism, AstraZeneca IMED Biotech Unit, Mölndal, Sweden.

Iterative screening has emerged as a promising approach to increase the efficiency of high-throughput screening (HTS) campaigns in drug discovery. By learning from a subset of the compound library, inferences on what compounds to screen next can be made by predictive models. One of the challenges of iterative screening is to decide how many iterations to perform. This is mainly related to difficulties in estimating the prospective hit rate in any given iteration. In this article, a novel method based on Venn - ABERS predictors is proposed. The method provides accurate estimates of the number of hits retrieved in any given iteration during an HTS campaign. The estimates provide the necessary information to support the decision on the number of iterations needed to maximize the screening outcome. Thus, this method offers a prospective screening strategy for early-stage drug discovery.

Åpne denne publikasjonen i ny fane eller vindu >>Are Traditional Neural Networks Well-Calibrated?### Johansson, Ulf

### Gabrielsson, Patrick

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_1_j_idt188_some",{id:"formSmash:j_idt184:1:j_idt188:some",widgetVar:"widget_formSmash_j_idt184_1_j_idt188_some",multiple:true}); PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_1_j_idt188_otherAuthors",{id:"formSmash:j_idt184:1:j_idt188:otherAuthors",widgetVar:"widget_formSmash_j_idt184_1_j_idt188_otherAuthors",multiple:true}); 2019 (engelsk)Inngår i: Proceedings of the International Joint Conference on Neural Networks, IEEE, 2019, Vol. July, artikkel-id 8851962Konferansepaper, Publicerat paper (Fagfellevurdert)
##### Abstract [en]

##### sted, utgiver, år, opplag, sider

IEEE, 2019
##### Emneord

Bagging, Calibration, Classification, Multilayer perceptrons, Probabilistic prediction, Venn-Abers predictors, Classification (of information), Multilayer neural networks, Multilayers, Best practices, Empirical investigation, Probabilistic classifiers, Sharp contrast
##### HSV kategori

##### Identifikatorer

urn:nbn:se:hj:diva-46689 (URN)10.1109/IJCNN.2019.8851962 (DOI)2-s2.0-85073208584 (Scopus ID)9781728119854 (ISBN)
##### Konferanse

2019 International Joint Conference on Neural Networks, IJCNN 2019, Budapest, Hungary, 14 - 19 July 2019
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_1_j_idt188_j_idt359",{id:"formSmash:j_idt184:1:j_idt188:j_idt359",widgetVar:"widget_formSmash_j_idt184_1_j_idt188_j_idt359",multiple:true});
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_1_j_idt188_j_idt365",{id:"formSmash:j_idt184:1:j_idt188:j_idt365",widgetVar:"widget_formSmash_j_idt184_1_j_idt188_j_idt365",multiple:true});
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_1_j_idt188_j_idt371",{id:"formSmash:j_idt184:1:j_idt188:j_idt371",widgetVar:"widget_formSmash_j_idt184_1_j_idt188_j_idt371",multiple:true});
#####

Tilgjengelig fra: 2019-10-25 Laget: 2019-10-25 Sist oppdatert: 2019-10-25bibliografisk kontrollert

Högskolan i Jönköping, Tekniska Högskolan, JTH, Datateknik och informatik, JTH, Jönköping AI Lab (JAIL).

Dept. of Information Technology, University of Borås, Sweden.

Traditional neural networks are generally considered to be well-calibrated. Consequently, the established best practice is to not try to improve the calibration using general techniques like Platt scaling. In this paper, it is demonstrated, using 25 publicly available two-class data sets, that both single multilayer perceptrons and ensembles of multilayer perceptrons in fact often are poorly calibrated. Furthermore, from the experimental results, it is obvious that the calibration can be significantly improved by using either Platt scaling or Venn-Abers predictors. These results stand in sharp contrast to the standard recommendations for the use of neural networks as probabilistic classifiers. The empirical investigation also shows that for bagged ensembles, it is beneficiary to calibrate on the out-of-bag instances, despite the fact that this leads to using substantially smaller ensembles for the predictions. Finally, an outright comparison between Platt scaling and Venn-Abers predictors shows that the latter most often produced significantly better calibrations, especially when calibrated on out-of-bag instances.

Åpne denne publikasjonen i ny fane eller vindu >>Calibrating probability estimation trees using Venn-Abers predictors### Johansson, Ulf

### Löfström, Tuve

Högskolan i Jönköping, Tekniska Högskolan, JTH, Datateknik och informatik, JTH, Jönköping AI Lab (JAIL).### Boström, Henrik

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_2_j_idt188_some",{id:"formSmash:j_idt184:2:j_idt188:some",widgetVar:"widget_formSmash_j_idt184_2_j_idt188_some",multiple:true}); PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_2_j_idt188_otherAuthors",{id:"formSmash:j_idt184:2:j_idt188:otherAuthors",widgetVar:"widget_formSmash_j_idt184_2_j_idt188_otherAuthors",multiple:true}); 2019 (engelsk)Inngår i: SIAM International Conference on Data Mining, SDM 2019, Society for Industrial and Applied Mathematics, 2019, s. 28-36Konferansepaper, Publicerat paper (Fagfellevurdert)
##### Abstract [en]

##### sted, utgiver, år, opplag, sider

Society for Industrial and Applied Mathematics, 2019
##### Emneord

Calibration, Data mining, Decision trees, Forestry, Laplace transforms, Calibration techniques, Class probabilities, Empirical investigation, Performance metrics, Predictive performance, Probability estimate, Probability estimation trees, Relative frequencies, Probability distributions
##### HSV kategori

##### Identifikatorer

urn:nbn:se:hj:diva-44355 (URN)10.1137/1.9781611975673.4 (DOI)2-s2.0-85066082095 (Scopus ID)9781611975673 (ISBN)
##### Konferanse

19th SIAM International Conference on Data Mining, SDM 2019, Hyatt Regency Calgary, Calgary, Canada, 2 - 4 May 2019
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_2_j_idt188_j_idt359",{id:"formSmash:j_idt184:2:j_idt188:j_idt359",widgetVar:"widget_formSmash_j_idt184_2_j_idt188_j_idt359",multiple:true});
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_2_j_idt188_j_idt365",{id:"formSmash:j_idt184:2:j_idt188:j_idt365",widgetVar:"widget_formSmash_j_idt184_2_j_idt188_j_idt365",multiple:true});
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_2_j_idt188_j_idt371",{id:"formSmash:j_idt184:2:j_idt188:j_idt371",widgetVar:"widget_formSmash_j_idt184_2_j_idt188_j_idt371",multiple:true});
#####

Tilgjengelig fra: 2019-06-11 Laget: 2019-06-11 Sist oppdatert: 2019-08-22bibliografisk kontrollert

Högskolan i Jönköping, Tekniska Högskolan, JTH, Datateknik och informatik, JTH, Jönköping AI Lab (JAIL).

School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Sweden.

Class labels output by standard decision trees are not very useful for making informed decisions, e.g., when comparing the expected utility of various alternatives. In contrast, probability estimation trees (PETs) output class probability distributions rather than single class labels. It is well known that estimating class probabilities in PETs by relative frequencies often lead to extreme probability estimates, and a number of approaches to provide more well-calibrated estimates have been proposed. In this study, a recent model-agnostic calibration approach, called Venn-Abers predictors is, for the first time, considered in the context of decision trees. Results from a large-scale empirical investigation are presented, comparing the novel approach to previous calibration techniques with respect to several different performance metrics, targeting both predictive performance and reliability of the estimates. All approaches are considered both with and without Laplace correction. The results show that using Venn-Abers predictors for calibration is a highly competitive approach, significantly outperforming Platt scaling, Isotonic regression and no calibration, with respect to almost all performance metrics used, independently of whether Laplace correction is applied or not. The only exception is AUC, where using non-calibrated PETs together with Laplace correction, actually is the best option, which can be explained by the fact that AUC is not affected by the absolute, but only relative, values of the probability estimates.

Åpne denne publikasjonen i ny fane eller vindu >>Customized interpretable conformal regressors### Johansson, Ulf

Högskolan i Jönköping, Tekniska Högskolan, JTH, Datateknik och informatik, JTH, Jönköping AI Lab (JAIL).### Sonstrod, C.

### Löfström, Tuwe

Högskolan i Jönköping, Tekniska Högskolan, JTH, Datateknik och informatik, JTH, Jönköping AI Lab (JAIL).### Bostrom, H.

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_3_j_idt188_some",{id:"formSmash:j_idt184:3:j_idt188:some",widgetVar:"widget_formSmash_j_idt184_3_j_idt188_some",multiple:true}); PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_3_j_idt188_otherAuthors",{id:"formSmash:j_idt184:3:j_idt188:otherAuthors",widgetVar:"widget_formSmash_j_idt184_3_j_idt188_otherAuthors",multiple:true}); 2019 (engelsk)Inngår i: Proceedings - 2019 IEEE International Conference on Data Science and Advanced Analytics, DSAA 2019, Institute of Electrical and Electronics Engineers (IEEE), 2019, s. 221-230, artikkel-id 8964179Konferansepaper, Publicerat paper (Fagfellevurdert)
##### Abstract [en]

##### sted, utgiver, år, opplag, sider

Institute of Electrical and Electronics Engineers (IEEE), 2019
##### Emneord

Conformal prediction, Interpretability, Oracle coaching, Predictive modeling, Regression trees, Advanced Analytics, Efficiency, Forecasting, Forestry, Labeled data, Regression analysis, Conformal predictions, Trees (mathematics)
##### HSV kategori

##### Identifikatorer

urn:nbn:se:hj:diva-47936 (URN)10.1109/DSAA.2019.00037 (DOI)2-s2.0-85079278508 (Scopus ID)9781728144931 (ISBN)
##### Konferanse

6th IEEE International Conference on Data Science and Advanced Analytics, DSAA 2019, Washington, United States, 5 - 8 October, 2019
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_3_j_idt188_j_idt359",{id:"formSmash:j_idt184:3:j_idt188:j_idt359",widgetVar:"widget_formSmash_j_idt184_3_j_idt188_j_idt359",multiple:true});
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_3_j_idt188_j_idt365",{id:"formSmash:j_idt184:3:j_idt188:j_idt365",widgetVar:"widget_formSmash_j_idt184_3_j_idt188_j_idt365",multiple:true});
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_3_j_idt188_j_idt371",{id:"formSmash:j_idt184:3:j_idt188:j_idt371",widgetVar:"widget_formSmash_j_idt184_3_j_idt188_j_idt371",multiple:true});
#####

##### Merknad

Dept. of Information Technology, University of Borås, Sweden.

School of Electrical Engineering and Computer Science, Kth Royal Institute of Technology, Sweden.

Interpretability is recognized as a key property of trustworthy predictive models. Only interpretable models make it straightforward to explain individual predictions, and allow inspection and analysis of the model itself. In real-world scenarios, these explanations and insights are often needed for a specific batch of predictions, i.e., a production set. If the input vectors for this production set are available when generating the predictive model, a methodology called oracle coaching can be used to produce highly accurate and interpretable models optimized for the specific production set. In this paper, oracle coaching is, for the first time, combined with the conformal prediction framework for predictive regression. A conformal regressor, which is built on top of a standard regression model, outputs valid prediction intervals, i.e., the error rate on novel data is bounded by a preset significance level, as long as the labeled data used for calibration is exchangeable with production data. Since validity is guaranteed for all conformal predictors, the key performance metric is the size of the prediction intervals, where tighter (more efficient) intervals are preferred. The efficiency of a conformal model depends on several factors, but more accurate underlying models will generally also lead to improved efficiency in the corresponding conformal predictor. A key contribution in this paper is the design of setups ensuring that when oracle coached regression trees, that per definition utilize knowledge about production data, are used as underlying models for conformal regressors, these remain valid. The experiments, using 20 publicly available regression data sets, demonstrate the validity of the suggested setups. Results also show that utilizing oracle-coached underlying models will generally lead to significantly more efficient conformal regressors, compared to when these are built on top of models induced using only training data.

This work was supported by the Swedish Knowledge Foundation (DATAKIND 20190194) and by Region Jönköping (DATAMINE HJ 2016/874-51).

Tilgjengelig fra: 2020-03-05 Laget: 2020-03-05 Sist oppdatert: 2020-03-05bibliografisk kontrollertÅpne denne publikasjonen i ny fane eller vindu >>Efficient conformal predictor ensembles### Linusson, Henrik

### Johansson, Ulf

Högskolan i Jönköping, Tekniska Högskolan, JTH, Datateknik och informatik, JTH, Jönköping AI Lab (JAIL).### Boström, Henrik

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_4_j_idt188_some",{id:"formSmash:j_idt184:4:j_idt188:some",widgetVar:"widget_formSmash_j_idt184_4_j_idt188_some",multiple:true}); PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_4_j_idt188_otherAuthors",{id:"formSmash:j_idt184:4:j_idt188:otherAuthors",widgetVar:"widget_formSmash_j_idt184_4_j_idt188_otherAuthors",multiple:true}); 2019 (engelsk)Inngår i: Neurocomputing, ISSN 0925-2312, E-ISSN 1872-8286Artikkel i tidsskrift (Fagfellevurdert) Epub ahead of print
##### Abstract [en]

##### sted, utgiver, år, opplag, sider

Elsevier, 2019
##### Emneord

Classification, Conformal prediction, Ensembles, Classification (of information), Forecasting, Statistical methods, Conformal predictions, Conformal predictors, Ensemble strategies, Error rate, P-values, Region size, Calibration, article, bootstrapping, prediction, theoretical study, validity
##### HSV kategori

##### Identifikatorer

urn:nbn:se:hj:diva-47219 (URN)10.1016/j.neucom.2019.07.113 (DOI)2-s2.0-85076549331 (Scopus ID)
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_4_j_idt188_j_idt359",{id:"formSmash:j_idt184:4:j_idt188:j_idt359",widgetVar:"widget_formSmash_j_idt184_4_j_idt188_j_idt359",multiple:true});
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_4_j_idt188_j_idt365",{id:"formSmash:j_idt184:4:j_idt188:j_idt365",widgetVar:"widget_formSmash_j_idt184_4_j_idt188_j_idt365",multiple:true});
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_4_j_idt188_j_idt371",{id:"formSmash:j_idt184:4:j_idt188:j_idt371",widgetVar:"widget_formSmash_j_idt184_4_j_idt188_j_idt371",multiple:true});
#####

##### Forskningsfinansiär

Knowledge Foundation, 20150185
Tilgjengelig fra: 2020-01-02 Laget: 2020-01-02 Sist oppdatert: 2020-01-02

Department of Information Technology, University of Borås, Sweden.

School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Sweden.

In this paper, we study a generalization of a recently developed strategy for generating conformal predictor ensembles: out-of-bag calibration. The ensemble strategy is evaluated, both theoretically and empirically, against a commonly used alternative ensemble strategy, bootstrap conformal prediction, as well as common non-ensemble strategies. A thorough analysis is provided of out-of-bag calibration, with respect to theoretical validity, empirical validity (error rate), efficiency (prediction region size) and p-value stability (the degree of variance observed over multiple predictions for the same object). Empirical results show that out-of-bag calibration displays favorable characteristics with regard to these criteria, and we propose that out-of-bag calibration be adopted as a standard method for constructing conformal predictor ensembles.

Åpne denne publikasjonen i ny fane eller vindu >>Efficient Venn Predictors using Random Forests### Johansson, Ulf

Högskolan i Jönköping, Tekniska Högskolan, JTH, Datateknik och informatik, JTH, Jönköping AI Lab (JAIL).### Löfström, Tuve

Högskolan i Jönköping, Tekniska Högskolan, JTH, Datateknik och informatik, JTH, Jönköping AI Lab (JAIL).### Linusson, Henrik

### Boström, Henrik

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_5_j_idt188_some",{id:"formSmash:j_idt184:5:j_idt188:some",widgetVar:"widget_formSmash_j_idt184_5_j_idt188_some",multiple:true}); PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_5_j_idt188_otherAuthors",{id:"formSmash:j_idt184:5:j_idt188:otherAuthors",widgetVar:"widget_formSmash_j_idt184_5_j_idt188_otherAuthors",multiple:true}); 2019 (engelsk)Inngår i: Machine Learning, ISSN 0885-6125, E-ISSN 1573-0565, Vol. 108, nr 3, s. 535-550Artikkel i tidsskrift (Fagfellevurdert) Published
##### Abstract [en]

##### sted, utgiver, år, opplag, sider

Springer, 2019
##### Emneord

Probabilistic prediction, Venn predictors, Venn-Abers predictors, Random forests, Out-of-bag calibration
##### HSV kategori

##### Identifikatorer

urn:nbn:se:hj:diva-41127 (URN)10.1007/s10994-018-5753-x (DOI)000459945900008 ()2-s2.0-85052523706 (Scopus ID)HOA JTH 2019 (Lokal ID)HOA JTH 2019 (Arkivnummer)HOA JTH 2019 (OAI)
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_5_j_idt188_j_idt359",{id:"formSmash:j_idt184:5:j_idt188:j_idt359",widgetVar:"widget_formSmash_j_idt184_5_j_idt188_j_idt359",multiple:true});
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_5_j_idt188_j_idt365",{id:"formSmash:j_idt184:5:j_idt188:j_idt365",widgetVar:"widget_formSmash_j_idt184_5_j_idt188_j_idt365",multiple:true});
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_5_j_idt188_j_idt371",{id:"formSmash:j_idt184:5:j_idt188:j_idt371",widgetVar:"widget_formSmash_j_idt184_5_j_idt188_j_idt371",multiple:true});
#####

Tilgjengelig fra: 2018-08-13 Laget: 2018-08-13 Sist oppdatert: 2019-08-22bibliografisk kontrollert

Högskolan i Borås, Department of Information Technology, Borås, Sweden.

The Royal Institute of Technology (KTH), School of Electrical Engineering and Computer Science, Stockholm, Sweden.

Successful use of probabilistic classification requires well-calibrated probability estimates, i.e., the predicted class probabilities must correspond to the true probabilities. In addition, a probabilistic classifier must, of course, also be as accurate as possible. In this paper, Venn predictors, and its special case Venn-Abers predictors, are evaluated for probabilistic classification, using random forests as the underlying models. Venn predictors output multiple probabilities for each label, i.e., the predicted label is associated with a probability interval. Since all Venn predictors are valid in the long run, the size of the probability intervals is very important, with tighter intervals being more informative. The standard solution when calibrating a classifier is to employ an additional step, transforming the outputs from a classifier into probability estimates, using a labeled data set not employed for training of the models. For random forests, and other bagged ensembles, it is, however, possible to use the out-of-bag instances for calibration, making all training data available for both model learning and calibration. This procedure has previously been successfully applied to conformal prediction, but was here evaluated for the first time for Venn predictors. The empirical investigation, using 22 publicly available data sets, showed that all four versions of the Venn predictors were better calibrated than both the raw estimates from the random forest, and the standard techniques Platt scaling and isotonic regression. Regarding both informativeness and accuracy, the standard Venn predictor calibrated on out-of-bag instances was the best setup evaluated. Most importantly, calibrating on out-of-bag instances, instead of using a separate calibration set, resulted in tighter intervals and more accurate models on every data set, for both the Venn predictors and the Venn-Abers predictors.

Åpne denne publikasjonen i ny fane eller vindu >>Interpretable and Specialized Conformal Predictors### Johansson, Ulf

Högskolan i Jönköping, Tekniska Högskolan, JTH, Datateknik och informatik, JTH, Jönköping AI Lab (JAIL).### Löfström, Tuwe

Högskolan i Jönköping, Tekniska Högskolan, JTH, Datateknik och informatik, JTH, Jönköping AI Lab (JAIL).### Boström, Henrik

### Sönströd, Cecilia

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_6_j_idt188_some",{id:"formSmash:j_idt184:6:j_idt188:some",widgetVar:"widget_formSmash_j_idt184_6_j_idt188_some",multiple:true}); PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_6_j_idt188_otherAuthors",{id:"formSmash:j_idt184:6:j_idt188:otherAuthors",widgetVar:"widget_formSmash_j_idt184_6_j_idt188_otherAuthors",multiple:true}); 2019 (engelsk)Inngår i: Conformal and Probabilistic Prediction and Applications / [ed] Alex Gammerman, Vladimir Vovk, Zhiyuan Luo, Evgueni Smirnov, 2019, s. 3-22Konferansepaper, Publicerat paper (Fagfellevurdert)
##### Abstract [en]

##### Serie

Proceedings of Machine Learning Research, ISSN 2640-3498 ; 105
##### Emneord

Interpretability, Decision trees, Classification, Oracle coaching, Conformal prediction
##### HSV kategori

##### Identifikatorer

urn:nbn:se:hj:diva-46804 (URN)
##### Konferanse

Proceedings of the Eighth Symposium on Conformal and Probabilistic Prediction and Applications, 9-11 September 2019, Golden Sands, Bulgaria
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_6_j_idt188_j_idt359",{id:"formSmash:j_idt184:6:j_idt188:j_idt359",widgetVar:"widget_formSmash_j_idt184_6_j_idt188_j_idt359",multiple:true});
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_6_j_idt188_j_idt365",{id:"formSmash:j_idt184:6:j_idt188:j_idt365",widgetVar:"widget_formSmash_j_idt184_6_j_idt188_j_idt365",multiple:true});
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_6_j_idt188_j_idt371",{id:"formSmash:j_idt184:6:j_idt188:j_idt371",widgetVar:"widget_formSmash_j_idt184_6_j_idt188_j_idt371",multiple:true});
#####

Tilgjengelig fra: 2019-11-11 Laget: 2019-11-11 Sist oppdatert: 2019-11-11bibliografisk kontrollert

School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Sweden.

Dept. of Information Technology, University of Borås, Sweden.

In real-world scenarios, interpretable models are often required to explain predictions, and to allow for inspection and analysis of the model. The overall purpose of oracle coaching is to produce highly accurate, but interpretable, models optimized for a specific test set. Oracle coaching is applicable to the very common scenario where explanations and insights are needed for a specific batch of predictions, and the input vectors for this test set are available when building the predictive model. In this paper, oracle coaching is used for generating underlying classifiers for conformal prediction. The resulting conformal classifiers output valid label sets, i.e., the error rate on the test data is bounded by a preset significance level, as long as the labeled data used for calibration is exchangeable with the test set. Since validity is guaranteed for all conformal predictors, the key performance metric is efficiency, i.e., the size of the label sets, where smaller sets are more informative. The main contribution of this paper is the design of setups making sure that when oracle-coached decision trees, that per definition utilize knowledge about test data, are used as underlying models for conformal classifiers, the exchangeability between calibration and test data is maintained. Consequently, the resulting conformal classifiers retain the validity guarantees. In the experimentation, using a large number of publicly available data sets, the validity of the suggested setups is empirically demonstrated. Furthermore, the results show that the more accurate underlying models produced by oracle coaching also improved the efficiency of the corresponding conformal classifiers.

Åpne denne publikasjonen i ny fane eller vindu >>Predicting with Confidence from Survival Data### Boström, Henrik

### Johansson, Ulf

Högskolan i Jönköping, Tekniska Högskolan, JTH, Datateknik och informatik, JTH, Jönköping AI Lab (JAIL).### Vesterberg, Anders

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_7_j_idt188_some",{id:"formSmash:j_idt184:7:j_idt188:some",widgetVar:"widget_formSmash_j_idt184_7_j_idt188_some",multiple:true}); PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_7_j_idt188_otherAuthors",{id:"formSmash:j_idt184:7:j_idt188:otherAuthors",widgetVar:"widget_formSmash_j_idt184_7_j_idt188_otherAuthors",multiple:true}); 2019 (engelsk)Inngår i: Conformal and Probabilistic Prediction and Applications / [ed] Alex Gammerman, Vladimir Vovk, Zhiyuan Luo, Evgueni Smirnov, 2019, s. 123-141Konferansepaper, Publicerat paper (Fagfellevurdert)
##### Abstract [en]

##### Serie

Proceedings of Machine Learning Research, ISSN 2640-3498 ; 105
##### Emneord

Conformal prediction, survival modeling, random forests.
##### HSV kategori

##### Identifikatorer

urn:nbn:se:hj:diva-46802 (URN)
##### Konferanse

Proceedings of the Eighth Symposium on Conformal and Probabilistic Prediction and Applications, 9-11 September 2019, Golden Sands, Bulgaria
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_7_j_idt188_j_idt359",{id:"formSmash:j_idt184:7:j_idt188:j_idt359",widgetVar:"widget_formSmash_j_idt184_7_j_idt188_j_idt359",multiple:true});
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_7_j_idt188_j_idt365",{id:"formSmash:j_idt184:7:j_idt188:j_idt365",widgetVar:"widget_formSmash_j_idt184_7_j_idt188_j_idt365",multiple:true});
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_7_j_idt188_j_idt371",{id:"formSmash:j_idt184:7:j_idt188:j_idt371",widgetVar:"widget_formSmash_j_idt184_7_j_idt188_j_idt371",multiple:true});
#####

Tilgjengelig fra: 2019-11-11 Laget: 2019-11-11 Sist oppdatert: 2019-11-11bibliografisk kontrollert

School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Sweden.

Scania CV AB, Sweden.

Survival modeling concerns predicting whether or not an event will occur before or on a given point in time. In a recent study, the conformal prediction framework was applied to this task, and so-called conformal random survival forest was proposed. It was empirically shown that the error level of this model indeed is very close to the provided confidence level, and also that the error for predicting each outcome, i.e., event or no-event, can be controlled separately by employing a Mondrian approach. The addressed task concerned making predictions for time points as provided by the underlying distribution. However, if one instead is interested in making predictions with respect to some specific time point, the guarantee of the conformal prediction framework no longer holds, as one is effectively considering a sample from another distribution than from which the calibration instances have been drawn. In this study, we propose a modification of the approach for specific time points, which transforms the problem into a binary classification task, thereby allowing the error level to be controlled. The latter is demonstrated by an empirical investigation using both a collection of publicly available datasets and two in-house datasets from a truck manufacturing company.

Åpne denne publikasjonen i ny fane eller vindu >>A data-driven approach to online fitting services### Löfström, Tuwe

### Johansson, Ulf

### Balkow, Jenny

### Sundell, Håkan

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_8_j_idt188_some",{id:"formSmash:j_idt184:8:j_idt188:some",widgetVar:"widget_formSmash_j_idt184_8_j_idt188_some",multiple:true}); PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_8_j_idt188_otherAuthors",{id:"formSmash:j_idt184:8:j_idt188:otherAuthors",widgetVar:"widget_formSmash_j_idt184_8_j_idt188_otherAuthors",multiple:true}); 2018 (engelsk)Inngår i: Data Science And Knowledge Engineering For Sensing Decision Support / [ed] Liu, J, Lu, J, Xu, Y, Martinez, L & Kerre, EE, World Scientific, 2018, Vol. 11, s. 1559-1566Konferansepaper, Publicerat paper (Fagfellevurdert)
##### Abstract [en]

##### sted, utgiver, år, opplag, sider

World Scientific, 2018
##### Serie

World Scientific Proceedings Series on Computer Engineering and Information Science ; 11
##### Emneord

Predictive regression; online fitting; fashion
##### HSV kategori

##### Identifikatorer

urn:nbn:se:hj:diva-44183 (URN)10.1142/9789813273238_0194 (DOI)000468160600194 ()978-981-3273-24-5 (ISBN)978-981-3273-22-1 (ISBN)
##### Konferanse

13th International Conference on Fuzzy Logic and Intelligent Technologies in Nuclear Science (FLINS), Belfast, Ireland, 21-24 August, 2018
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_8_j_idt188_j_idt359",{id:"formSmash:j_idt184:8:j_idt188:j_idt359",widgetVar:"widget_formSmash_j_idt184_8_j_idt188_j_idt359",multiple:true});
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_8_j_idt188_j_idt365",{id:"formSmash:j_idt184:8:j_idt188:j_idt365",widgetVar:"widget_formSmash_j_idt184_8_j_idt188_j_idt365",multiple:true});
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_8_j_idt188_j_idt371",{id:"formSmash:j_idt184:8:j_idt188:j_idt371",widgetVar:"widget_formSmash_j_idt184_8_j_idt188_j_idt371",multiple:true});
#####

Tilgjengelig fra: 2019-06-11 Laget: 2019-06-11 Sist oppdatert: 2019-08-22bibliografisk kontrollert

Högskolan i Jönköping, Tekniska Högskolan, JTH, Datateknik och informatik, JTH, Jönköping AI Lab (JAIL). Univ Boras, Dept Informat Technol, Boras, Sweden.

Högskolan i Jönköping, Tekniska Högskolan, JTH, Datateknik och informatik, JTH, Jönköping AI Lab (JAIL). Univ Boras, Dept Informat Technol, Boras, Sweden.

Univ Boras, Swedish Sch Text, Boras, Sweden.

Högskolan i Jönköping, Tekniska Högskolan, JTH, Datateknik och informatik, JTH, Jönköping AI Lab (JAIL). Univ Boras, Dept Informat Technol, Boras, Sweden.

Being able to accurately predict several attributes related to size is vital for services supporting online fitting. In this paper, we investigate a data-driven approach, while comparing two different supervised modeling techniques for predictive regression; standard multiple linear regression and neural networks. Using a fairly large, publicly available, data set of high quality, the main results are somewhat discouraging. Specifically, it is questionable whether key attributes like sleeve length, neck size, waist and chest can be modeled accurately enough using easily accessible input variables as sex, weight and height. This is despite the fact that several services online offer exactly this functionality. For this specific task, the results show that standard linear regression was as accurate as the potentially more powerful neural networks. Most importantly, comparing the predictions to reasonable levels for acceptable errors, it was found that an overwhelming majority of all instances had at least one attribute with an unacceptably high prediction error. In fact, if requiring that all variables are predicted with an acceptable accuracy, less than 5 % of all instances met that criterion. Specifically, for females, the success rate was as low as 1.8 %.

Åpne denne publikasjonen i ny fane eller vindu >>Classification with reject option using conformal prediction### Linusson, Henrik

### Johansson, Ulf

Högskolan i Jönköping, Tekniska Högskolan, JTH, Datateknik och informatik, JTH, Jönköping AI Lab (JAIL).### Boström, Henrik

### Löfström, Tuve

Högskolan i Jönköping, Tekniska Högskolan, JTH, Datateknik och informatik, JTH, Jönköping AI Lab (JAIL).PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_9_j_idt188_some",{id:"formSmash:j_idt184:9:j_idt188:some",widgetVar:"widget_formSmash_j_idt184_9_j_idt188_some",multiple:true}); PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_9_j_idt188_otherAuthors",{id:"formSmash:j_idt184:9:j_idt188:otherAuthors",widgetVar:"widget_formSmash_j_idt184_9_j_idt188_otherAuthors",multiple:true}); 2018 (engelsk)Inngår i: Advances in Knowledge Discovery and Data Mining: 22nd Pacific-Asia Conference, PAKDD 2018, Melbourne, VIC, Australia, June 3-6, 2018, Proceedings, Part I, Springer, 2018, s. 94-105Konferansepaper, Publicerat paper (Fagfellevurdert)
##### Abstract [en]

##### sted, utgiver, år, opplag, sider

Springer, 2018
##### Serie

Lecture Notes in Computer Science, ISSN 0302-9743 ; 10937
##### Emneord

Data mining, Errors, Forecasting, Testing, Uncertainty analysis, Benchmark datasets, Classification procedure, Conformal predictions, Cumulative errors, Empirical evaluations, Error rate, Test object, Test sets, Classification (of information)
##### HSV kategori

##### Identifikatorer

urn:nbn:se:hj:diva-41260 (URN)10.1007/978-3-319-93034-3_8 (DOI)000443224400008 ()2-s2.0-85049360232 (Scopus ID)9783319930336 (ISBN)
##### Konferanse

22nd Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, PAKDD 2018; Melbourne; Australia; 3 June 2018 through 6 June 2018
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_9_j_idt188_j_idt359",{id:"formSmash:j_idt184:9:j_idt188:j_idt359",widgetVar:"widget_formSmash_j_idt184_9_j_idt188_j_idt359",multiple:true});
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_9_j_idt188_j_idt365",{id:"formSmash:j_idt184:9:j_idt188:j_idt365",widgetVar:"widget_formSmash_j_idt184_9_j_idt188_j_idt365",multiple:true});
#####

PrimeFaces.cw("AccordionPanel","widget_formSmash_j_idt184_9_j_idt188_j_idt371",{id:"formSmash:j_idt184:9:j_idt188:j_idt371",widgetVar:"widget_formSmash_j_idt184_9_j_idt188_j_idt371",multiple:true});
#####

##### Forskningsfinansiär

Knowledge Foundation
Tilgjengelig fra: 2018-08-27 Laget: 2018-08-27 Sist oppdatert: 2019-08-22bibliografisk kontrollert

Department of Information Technology, University of Borås, Borås, Sweden.

School of Electrical Engineering and Computer Science, Royal Institute of Technology, Kista, Sweden.

In this paper, we propose a practically useful means of interpreting the predictions produced by a conformal classifier. The proposed interpretation leads to a classifier with a reject option, that allows the user to limit the number of erroneous predictions made on the test set, without any need to reveal the true labels of the test objects. The method described in this paper works by estimating the cumulative error count on a set of predictions provided by a conformal classifier, ordered by their confidence. Given a test set and a user-specified parameter k, the proposed classification procedure outputs the largest possible amount of predictions containing on average at most k errors, while refusing to make predictions for test objects where it is too uncertain. We conduct an empirical evaluation using benchmark datasets, and show that we are able to provide accurate estimates for the error rate on the test set.