Combining Classifiers with Informational Confidence

TitleCombining Classifiers with Informational Confidence
Publication TypeBook Chapters
Year of Publication2008
AuthorsJaeger S, Ma H, Doermann D
EditorSimone Marinai H F
Book TitleStudies in Computational Intelligence: Machine Learning in Document Analysis and RecognitionStudies in Computational Intelligence: Machine Learning in Document Analysis and Recognition
Pagination163 - 192

We propose a new statistical method for learning normalized confidence values in multiple classifier systems. Our main idea is to adjust confidence values so that their nominal values equal the information actually conveyed. In order to do so, we assume that information depends on the actual performance of each confidence value on an evaluation set. As information measure, we use Shannon's well-known logarithmic notion of information. With the confidence values matching their informational content, the classifier combination scheme reduces to the simple sum-rule, theoretically justifying this elementary combination scheme. In experimental evaluations for script identification, and both handwritten and printed character recognition, we achieve a consistent improvement on the best single recognition rate. We cherish the hope that our information-theoretical framework helps fill the theoretical gap we still experience in classifier combination, and puts the excellent practical performance of multiple classifier systems on a more solid basis.