Evaluation of a variance-based nonconformity measure for regression forests
2016 (English)In: Conformal and Probabilistic Prediction with Applications, Springer, 2016, p. 75-89Conference paper, Published paper (Refereed)
Abstract [en]
In a previous large-scale empirical evaluation of conformal regression approaches, random forests using out-of-bag instances for calibration together with a k-nearest neighbor-based nonconformity measure, was shown to obtain state-of-the-art performance with respect to efficiency, i.e., average size of prediction regions. However, the use of the nearest-neighbor procedure not only requires that all training data have to be retained in conjunction with the underlying model, but also that a significant computational overhead is incurred, during both training and testing. In this study, a more straightforward nonconformity measure is investigated, where the difficulty estimate employed for normalization is based on the variance of the predictions made by the trees in a forest. A large-scale empirical evaluation is presented, showing that both the nearest-neighbor-based and the variance-based measures significantly outperform a standard (non-normalized) nonconformity measure, while no significant difference in efficiency between the two normalized approaches is observed. Moreover, the evaluation shows that state-of-theart performance is achieved by the variance-based measure at a computational cost that is several orders of magnitude lower than when employing the nearest-neighbor-based nonconformity measure.
Place, publisher, year, edition, pages
Springer, 2016. p. 75-89
Series
Lecture Notes in Computer Science, ISSN 0302-9743 ; 9653
Keywords [en]
Conformal prediction, Nonconformity measures, Random forests, Regression, Decision trees, Efficiency, Nearest neighbor search, Regression analysis, Computational overheads, Conformal predictions, Empirical evaluations, State-of-the-art performance, Training and testing, Forecasting
National Category
Computer Sciences
Identifiers
URN: urn:nbn:se:hj:diva-38115DOI: 10.1007/978-3-319-33395-3_6Scopus ID: 2-s2.0-84964088557Local ID: 0;0;miljJAILISBN: 9783319333946 (print)OAI: oai:DiVA.org:hj-38115DiVA, id: diva2:1163934
Conference
5th International Symposium on Conformal and Probabilistic Prediction with Applications, COPA 2016; Madrid; Spain; 20 April 2016 through 22 April 2016
2017-12-082017-12-082019-08-23Bibliographically approved