Change search
Link to record
Permanent link

Direct link
BETA
Sönströd, Cecilia
Publications (8 of 8) Show all publications
Johansson, U., Löfström, T. & Sönströd, C. (2011). Locally Induced Predictive Models. In: : . Paper presented at IEEE International Conference on Systems, Man, and Cybernetics. IEEE
Open this publication in new window or tab >>Locally Induced Predictive Models
2011 (English)Conference paper, Published paper (Refereed)
Abstract [en]

Most predictive modeling techniques utilize all available data to build global models. This is despite the wellknown fact that for many problems, the targeted relationship varies greatly over the input space, thus suggesting that localized models may improve predictive performance. In this paper, we suggest and evaluate a technique inducing one predictive model for each test instance, using only neighboring instances. In the experimentation, several different variations of the suggested algorithm producing localized decision trees and neural network models are evaluated on 30 UCI data sets. The main result is that the suggested approach generally yields better predictive performance than global models built using all available training data. As a matter of fact, all techniques producing J48 trees obtained significantly higher accuracy and AUC, compared to the global J48 model. For RBF network models, with their inherent ability to use localized information, the suggested approach was only successful with regard to accuracy, while global RBF models had a better ranking ability, as seen by their generally higher AUCs.

Place, publisher, year, edition, pages
IEEE, 2011
Keywords
local learning, predictive modeling, decision trees, rbf networks, Machine Learning, Data Mining
National Category
Computer Sciences Computer and Information Sciences
Research subject
Business and IT
Identifiers
urn:nbn:se:hj:diva-45799 (URN)10.1109/ICSMC.2011.6083922 (DOI)0;0;miljJAIL (Local ID)978-1-4577-0651-6 (ISBN)0;0;miljJAIL (Archive number)0;0;miljJAIL (OAI)
Conference
IEEE International Conference on Systems, Man, and Cybernetics
Available from: 2019-09-06 Created: 2019-09-06 Last updated: 2019-09-06Bibliographically approved
Johansson, U., Sönströd, C. & Löfström, T. (2011). One Tree to Explain Them All. In: : . Paper presented at IEEE Congress on Evolutionary Computation (CEC). IEEE
Open this publication in new window or tab >>One Tree to Explain Them All
2011 (English)Conference paper, Published paper (Refereed)
Abstract [en]

Random forest is an often used ensemble technique, renowned for its high predictive performance. Random forests models are, however, due to their sheer complexity inherently opaque, making human interpretation and analysis impossible. This paper presents a method of approximating the random forest with just one decision tree. The approach uses oracle coaching, a recently suggested technique where a weaker but transparent model is generated using combinations of regular training data and test data initially labeled by a strong classifier, called the oracle. In this study, the random forest plays the part of the oracle, while the transparent models are decision trees generated by either the standard tree inducer J48, or by evolving genetic programs. Evaluation on 30 data sets from the UCI repository shows that oracle coaching significantly improves both accuracy and area under ROC curve, compared to using training data only. As a matter of fact, resulting single tree models are as accurate as the random forest, on the specific test instances. Most importantly, this is not achieved by inducing or evolving huge trees having perfect fidelity; a large majority of all trees are instead rather compact and clearly comprehensible. The experiments also show that the evolution outperformed J48, with regard to accuracy, but that this came at the expense of slightly larger trees.

Place, publisher, year, edition, pages
IEEE, 2011
Keywords
genetic programming, random forest, oracle coaching, decision trees, Machine learning, Data mining
National Category
Computer Sciences Computer and Information Sciences
Research subject
Business and IT
Identifiers
urn:nbn:se:hj:diva-45800 (URN)0;0;miljJAIL (Local ID)978-1-4244-7834-7 (ISBN)0;0;miljJAIL (Archive number)0;0;miljJAIL (OAI)
Conference
IEEE Congress on Evolutionary Computation (CEC)
Note

Sponsorship:

This work was supported by the INFUSIS project www.his.se/infusis at the University of Skövde, Sweden, in partnership with the Swedish Knowledge Foundation under grant 2008/0502.

Available from: 2019-09-06 Created: 2019-09-06 Last updated: 2019-09-06Bibliographically approved
Johansson, U., Sönströd, C. & Löfström, T. (2010). Oracle Coached Decision Trees and Lists. In: : . Paper presented at Advances in Intelligent Data Analysis IX, 9th International Symposium, IDA 2010. Springer
Open this publication in new window or tab >>Oracle Coached Decision Trees and Lists
2010 (English)Conference paper, Published paper (Refereed)
Abstract [en]

This paper introduces a novel method for obtaining increased predictive performance from transparent models in situations where production input vectors are available when building the model. First, labeled training data is used to build a powerful opaque model, called an oracle. Second, the oracle is applied to production instances, generating predicted target values, which are used as labels. Finally, these newly labeled instances are utilized, in different combinations with normal training data, when inducing a transparent model. Experimental results, on 26 UCI data sets, show that the use of oracle coaches significantly improves predictive performance, compared to standard model induction. Most importantly, both accuracy and AUC results are robust over all combinations of opaque and transparent models evaluated. This study thus implies that the straightforward procedure of using a coaching oracle, which can be used with arbitrary classifiers, yields significantly better predictive performance at a low computational cost.

Place, publisher, year, edition, pages
Springer, 2010
Series
LNCS ; 6065
Keywords
decision trees, rule learning, coaching, Machine learning
National Category
Computer Sciences Information Systems
Identifiers
urn:nbn:se:hj:diva-45803 (URN)10.1007/978-3-642-13062-5_8 (DOI)0;0;miljJAIL (Local ID)978-3-642-13061-8 (ISBN)0;0;miljJAIL (Archive number)0;0;miljJAIL (OAI)
Conference
Advances in Intelligent Data Analysis IX, 9th International Symposium, IDA 2010
Available from: 2019-09-06 Created: 2019-09-06 Last updated: 2019-09-06Bibliographically approved
Johansson, U., Sönströd, C., Norinder, U., Boström, H. & Löfström, T. (2010). Using Feature Selection with Bagging and Rule Extraction in Drug Discovery. In: : . Paper presented at Advances in Intelligent Decision Technologies, Second KES International Symposium IDT 2010. Springer
Open this publication in new window or tab >>Using Feature Selection with Bagging and Rule Extraction in Drug Discovery
Show others...
2010 (English)Conference paper, Published paper (Refereed)
Abstract [en]

This paper investigates different ways of combining feature selection with bagging and rule extraction in predictive modeling. Experiments on a large number of data sets from the medicinal chemistry domain, using standard algorithms implemented in theWeka data mining workbench, show that feature selection can lead to significantly improved predictive performance.When combining feature selection with bagging, employing the feature selection on each bootstrap obtains the best result.When using decision trees for rule extraction, the effect of feature selection can actually be detrimental, unless the transductive approach oracle coaching is also used. However, employing oracle coaching will lead to significantly improved performance, and the best results are obtainedwhen performing feature selection before training the opaque model. The overall conclusion is that it can make a substantial difference for the predictive performance exactly how feature selection is used in conjunction with other techniques.

Place, publisher, year, edition, pages
Springer, 2010
Series
Smart Innovation, Systems and Technologies ; 4
Keywords
feature selection, bagging, rule extraction, Machine learning
National Category
Computer Sciences Information Systems
Identifiers
urn:nbn:se:hj:diva-45804 (URN)0;0;miljJAIL (Local ID)978-3-642-14615-2 (ISBN)0;0;miljJAIL (Archive number)0;0;miljJAIL (OAI)
Conference
Advances in Intelligent Decision Technologies, Second KES International Symposium IDT 2010
Note

Sponsorship:

This work was supported by the INFUSIS project (www.his.se/infusis) at the University of Skövde, Sweden, in partnership with the Swedish Knowledge Foundation under grant 2008/0502.

Available from: 2019-09-06 Created: 2019-09-06 Last updated: 2019-09-06Bibliographically approved
Sönströd, C., Johansson, U. & Löfström, T. (2009). Evaluating Algorithms for Concept Description. In: : . Paper presented at 5th International Conference on Data Mining - DMIN 09, Las Vegas, USA. CSREA
Open this publication in new window or tab >>Evaluating Algorithms for Concept Description
2009 (English)Conference paper, Published paper (Refereed)
Abstract [en]

When performing concept description, models need to be evaluated both on accuracy and comprehensibility. A comprehensible concept description model should present the most important relationships in the data in an accurate and understandable way. Two natural representations for this are decision trees and decision lists. In this study, the two decision list algorithms RIPPER and Chipper, and the decision tree algorithm C4.5, are evaluated for concept description, using publicly available datasets. The experiments show that C4.5 performs very well regarding accuracy and brevity, i.e. the ability to classify instances with few tests, but also produces large models that are hard to survey and contain many extremely specific rules, thus not being good concept descriptions. The decision list algorithms perform reasonably well on accuracy, and are mostly able to produce small models with relatively good predictive performance. Regarding brevity, Chipper is better than RIPPER, using on average fewer conditions to classify an instance. RIPPER, on the other hand, excels in relevance, i.e. the ability to capture a large number of instances with every rule.

Place, publisher, year, edition, pages
CSREA, 2009
Keywords
concept description, rule induction, decision lists, Machine Learning, data mining
National Category
Computer Sciences Computer and Information Sciences
Identifiers
urn:nbn:se:hj:diva-45806 (URN)0;0;miljJAIL (Local ID)0;0;miljJAIL (Archive number)0;0;miljJAIL (OAI)
Conference
5th International Conference on Data Mining - DMIN 09, Las Vegas, USA
Available from: 2019-09-06 Created: 2019-09-06 Last updated: 2019-09-06Bibliographically approved
Johansson, U., König, R., Löfström, T., Sönströd, C. & Niklasson, L. (2009). Post-processing Evolved Decision Trees. In: Ajith Abraham (Ed.), Foundations of Computational Intelligence: (pp. 149-164). Springer
Open this publication in new window or tab >>Post-processing Evolved Decision Trees
Show others...
2009 (English)In: Foundations of Computational Intelligence / [ed] Ajith Abraham, Springer, 2009, p. 149-164Chapter in book (Other academic)
Abstract [en]

Although Genetic Programming (GP) is a very general technique, it is also quite powerful. As a matter of fact, GP has often been shown to outperform more specialized techniques on a variety of tasks. In data mining, GP has successfully been applied to most major tasks; e.g. classification, regression and clustering. In this chapter, we introduce, describe and evaluate a straightforward novel algorithm for post-processing genetically evolved decision trees. The algorithm works by iteratively, one node at a time, search for possible modifications that will result in higher accuracy. More specifically, the algorithm, for each interior test, evaluates every possible split for the current attribute and chooses the best. With this design, the post-processing algorithm can only increase training accuracy, never decrease it. In the experiments, the suggested algorithm is applied to GP decision trees, either induced directly from datasets, or extracted from neural network ensembles. The experimentation, using 22 UCI datasets, shows that the suggested post-processing technique results in higher test set accuracies on a large majority of the datasets. As a matter of fact, the increase in test accuracy is statistically significant for one of the four evaluated setups, and substantial on two out of the other three.

Place, publisher, year, edition, pages
Springer, 2009
Keywords
decision trees, genetic programming, Machine learning, data mining
National Category
Computer and Information Sciences Computer and Information Sciences
Identifiers
urn:nbn:se:hj:diva-45808 (URN)10.1007/978-3-642-01088-0 (DOI)0;0;miljJAIL (Local ID)978-3-642-01087-3 (ISBN)0;0;miljJAIL (Archive number)0;0;miljJAIL (OAI)
Available from: 2015-12-17 Created: 2019-09-06Bibliographically approved
Johansson, U., Sönströd, C., Löfström, T. & König, R. (2009). Using Genetic Programming to Obtain Implicit Diversity. In: : . Paper presented at 2009 IEEE Congress on Evolutionary Computation (CEC 2009), Trondheim, Norge. IEEE
Open this publication in new window or tab >>Using Genetic Programming to Obtain Implicit Diversity
2009 (English)Conference paper, Published paper (Refereed)
Abstract [en]

When performing predictive data mining, the use of ensembles is known to increase prediction accuracy, compared to single models. To obtain this higher accuracy, ensembles should be built from base classifiers that are both accurate and diverse. The question of how to balance these two properties in order to maximize ensemble accuracy is, however, far from solved and many different techniques for obtaining ensemble diversity exist. One such technique is bagging, where implicit diversity is introduced by training base classifiers on different subsets of available data instances, thus resulting in less accurate, but diverse base classifiers. In this paper, genetic programming is used as an alternative method to obtain implicit diversity in ensembles by evolving accurate, but different base classifiers in the form of decision trees, thus exploiting the inherent inconsistency of genetic programming. The experiments show that the GP approach outperforms standard bagging of decision trees, obtaining significantly higher ensemble accuracy over 25 UCI datasets. This superior performance stems from base classifiers having both higher average accuracy and more diversity. Implicitly introducing diversity using GP thus works very well, since evolved base classifiers tend to be highly accurate and diverse.

Place, publisher, year, edition, pages
IEEE, 2009
Keywords
genetic programming, bagging, ensembles, diversity, Machine learning
National Category
Computer and Information Sciences Computer and Information Sciences
Identifiers
urn:nbn:se:hj:diva-45809 (URN)0;0;miljJAIL (Local ID)978-1-4244-2959-2 (ISBN)0;0;miljJAIL (Archive number)0;0;miljJAIL (OAI)
Conference
2009 IEEE Congress on Evolutionary Computation (CEC 2009), Trondheim, Norge
Available from: 2019-09-06 Created: 2019-09-06 Last updated: 2019-09-06Bibliographically approved
Johansson, U., Sönströd, C., Boström, H. & Löfström, T. (2008). Chipper: A Novel Algorithm for Concept Description. In: : . Paper presented at Paper presented at the 10th Scandinavian Conference on Artificial Intelligence SCAI 2008. IOS Press
Open this publication in new window or tab >>Chipper: A Novel Algorithm for Concept Description
2008 (English)Conference paper, Published paper (Refereed)
Abstract [en]

In this paper, several demands placed on concept description algorithms are identified and discussed. The most important criterion is the ability to produce compact rule sets that, in a natural and accurate way, describe the most important relationships in the underlying domain. An algorithm based on the identified criteria is presented and evaluated. The algorithm, named Chipper, produces decision lists, where each rule covers a maximum number of remaining instances while meeting requested accuracy requirements. In the experiments, Chipper is evaluated on nine UCI data sets. The main result is that Chipper produces compact and understandable rule sets, clearly fulfilling the overall goal of concept description. In the experiments, Chipper's accuracy is similar to standard decision tree and rule induction algorithms, while rule sets have superior comprehensibility.

Place, publisher, year, edition, pages
IOS Press, 2008
Keywords
concept description, decision lists, nachine learning, Machine Learning, Data Mining, Computer Science, data mining
National Category
Computer and Information Sciences Information Systems
Identifiers
urn:nbn:se:hj:diva-45812 (URN)0;0;miljJAIL (Local ID)978-1-58603-867-0 (ISBN)0;0;miljJAIL (Archive number)0;0;miljJAIL (OAI)
Conference
Paper presented at the 10th Scandinavian Conference on Artificial Intelligence SCAI 2008
Available from: 2019-09-06 Created: 2019-09-06 Last updated: 2019-09-06Bibliographically approved
Organisations

Search in DiVA

Show all publications