Feature selection using localized generalization error for supervised classification problems using RBFNN

Authors

W. Y., Wing NG, School of Computer Science and Engineering, South China University of Technology, China; Department of Computing, The Hong Kong Polytechnic University, Hong Kong
Daniel S. YEUNG, School of Computer Science and Engineering, South China University of Technology, China; Department of Computing, The Hong Kong Polytechnic University, Hong Kong
Michael FIRTH, Department of Finance and Insurance, Lingnan University, Hong Kong
C. C, Eric TSANG, Department of Computing, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong
Xi Zhao WANG, Machine Learning Center, Faculty of Mathematics and Computer Science, Hebei University, China

Document Type

Journal article

Source Publication

Pattern Recognition

Publication Date

12-1-2008

Volume

41

Issue

12

First Page

3706

Last Page

3719

Keywords

Feature selection, Generalization error, Neural network, RBFNN

Abstract

A pattern classification problem usually involves using high-dimensional features that make the classifier very complex and difficult to train. With no feature reduction, both training accuracy and generalization capability will suffer. This paper proposes a novel hybrid filter-wrapper-type feature subset selection methodology using a localized generalization error model. The localized generalization error model for a radial basis function neural network bounds from above the generalization error for unseen samples located within a neighborhood of the training samples. Iteratively, the feature making the smallest contribution to the generalization error bound is removed. Moreover, the novel feature selection method is independent of the sample size and is computationally fast. The experimental results show that the proposed method consistently removes large percentages of features with statistically insignificant loss of testing accuracy for unseen samples. In the experiments for two of the datasets, the classifiers built using feature subsets with 90% of features removed by our proposed approach yield average testing accuracies higher than those trained using the full set of features. Finally, we corroborate the efficacy of the model by using it to predict corporate bankruptcies in the US.

DOI

10.1016/j.patcog.2008.05.004

Print ISSN

00313203

Funding Information

This work is supported by the Hong Kong Polytechnic University Research Grant G-YD87. {G-YD87}

Publisher Statement

Copyright © 2008 Elsevier Ltd. All rights reserved. Access to external full text or publisher's version may require subscription.

Full-text Version

Publisher’s Version

Language

English

Recommended Citation

Ng, W. W. Y., Yeung, D. S., Firth, M., Tsang, E. C. C., & Wang, X.-Z. (2008). Feature selection using localized generalization error for supervised classification problems using RBFNN. Pattern Recognition, 41(12), 3706-3719. doi: 10.1016/j.patcog.2008.05.004