Advanced SearchSearch Tips
Improving an Ensemble Model by Optimizing Bootstrap Sampling
facebook(new window)  Pirnt(new window) E-mail(new window) Excel Download
 Title & Authors
Improving an Ensemble Model by Optimizing Bootstrap Sampling
Min, Sung-Hwan;
  PDF(new window)
Ensemble classification involves combining multiple classifiers to obtain more accurate predictions than those obtained using individual models. Ensemble learning techniques are known to be very useful for improving prediction accuracy. Bagging is one of the most popular ensemble learning techniques. Bagging has been known to be successful in increasing the accuracy of prediction of the individual classifiers. Bagging draws bootstrap samples from the training sample, applies the classifier to each bootstrap sample, and then combines the predictions of these classifiers to get the final classification result. Bootstrap samples are simple random samples selected from the original training data, so not all bootstrap samples are equally informative, due to the randomness. In this study, we proposed a new method for improving the performance of the standard bagging ensemble by optimizing bootstrap samples. A genetic algorithm is used to optimize bootstrap samples of the ensemble for improving prediction accuracy of the ensemble model. The proposed model is applied to a bankruptcy prediction problem using a real dataset from Korean companies. The experimental results showed the effectiveness of the proposed model.
Bagging;Bankruptcy Prediction;Ensemble;Genetic Algorithms;
 Cited by
T. G. Dietterich, "Machine-learning research: Four current directions," AI Magazine, Vol.18, No.4, 1997, pp. 97-136. crossref(new window)

L. I. Kuncheva, Combining Pattern Classifiers: Methods and Algorithms, John Wiley & Sons, Inc., Hoboken, New Jersey, 2004. Methods-Algorithms/dp/0471210781

S. Bian, W. Wang, "On diversity and accuracy of homogeneous and heterogeneous ensembles," International Journal of Hybrid Intelligent Systems, Vol.4, No.2, 2007, pp.103-128. f-hybrid-intelligent-systems/his00044 crossref(new window)

L. I. Kuncheva, C. J. Whitaker, "Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy," Machine Learning, Vol.51, No.2, 2003, pp. 181-207. crossref(new window)

L. Breiman, "Bagging predictors," Machine Learning, Vol. 24, No.2, 1996, pp. 123-140. crossref(new window)

Y. Freund, R. Schapire, "Experiments with a new boosting algorithm," Proceedings of the 13th International Conference on Machine learning, 1996, pp. 148-156. 33.1040

T. Ho, "The random subspace method for construction decision forests," IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, pp.832-844. crossref(new window)

L. Hansen, Salamon, P, "Neural network ensembles," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.12, No.10, 1990, pp. 993-1001. crossref(new window)

S.H. Kim, J.W., Kim, "SOHO Bankruptcy Prediction Using Modified Bagging Predictors," Journal of Intelligence and Information Systems, Vol.13, No.2, 2007, pp. 15-26. =JJSHBB_2007_v13n2_15

C. Tsai, J. Wu. "Using Neural Network Ensembles for Bankruptcy Prediction and Credit Scoring," Expert Systems with Applications, Vol.34, No.4, 2008, pp. 2639-2649. crossref(new window)

M. Kim, "A Performance Comparison of Ensemble in Bankruptcy Prediction," Entrue Journal of Information Technology, Vol.8, No.2, 2009, pp. 41-49.

H. Li, Y.-C. Lee, Y.-C. Zhou, J. Sun, "The random subspace binary logit (RSBL) model for bankruptcy prediction," Knowledge-Based Systems, Vol. 24, No.8, 2011, pp. 1380-1388 crossref(new window)

S. Min, "Developing an Ensemble Classifier for Bankruptcy Prediction," Journal of the Korea Industrial Information Systems Research, Vol. 17, No. 7, 2012, pp. 139-148. crossref(new window)

A.I. Marques, V. Garcia, and J. S. Sanchez. "Two-Level Classifier Ensembles for Credit Risk Assessment," Expert Systems with Applications, Vol.39, No.12, 2012, pp. 10916-10922. crossref(new window)

H.N. Choi, D.H. Lim, "Bankruptcy prediction using ensemble SVM model," Journal of the Korean Data and Information Science Society, Vol.24, No.6, 2013, 1113-1125. crossref(new window)

S. Min, "Bankruptcy Prediction Using an Improved Bagging Ensemble," Journal of Intelligence and Information Systems, Vol.20, No.4, 2014, pp. 121-139. crossref(new window)

M. Kim, D. Kang, H.B. Kim, "Geometric Mean Based Boosting Algorithm with over-Sampling to Resolve Data Imbalance Problem for Bankruptcy Prediction," Expert Systems with Applications, Vol.42, No.3, 2015, pp. 1074-1082. crossref(new window)

C. Hung, J-H. Chen, "A Selective Ensemble Based on Expected Probabilities for Bankruptcy Prediction," Expert Systems with Applications, Vol.36, No.3, 2009, pp. 5297-5303. crossref(new window)

K. Li, Z. Liu, Y. Han, "Study of Selective Ensemble Learning Methods Based on Support Vector Machine," Physics Procedia, Vol. 33, 2012, pp.1518-1525. crossref(new window)

Y. Guo, et al., "A Novel Dynamic Rough Subspace Based Selective Ensemble," Pattern Recognition, Vol.48, No.5, 2014, pp. 1638-1652. crossref(new window)

T. K. Ho, "Multiple classifier combination: Lessons and the next steps," In A. Kandel and H. Bunke, editors, Hybrid Methods in Pattern Recognition. World Scientific Publishing, 2002 crossref(new window)

D. E. Goldberg, "Genetic algorithms in search, optimization and machine learning," New York: Addison-Wesley, 1989. c-Algorithms-in-Search-Optimization-and-Machine-Lear ning/