Class membership probabilities
In general problems of classification, class membership probabilities reflect the uncertainty with which a given individual item can be assigned to any given class. Applications of classification in machine learning usually supply membership values that do not induce any probabilistic confidence. It is desirable, to transform or re-scale membership values to class membership probabilities, since they are comparable and additionally are more easily applicable for post-processing.
There exist several univariate calibration methods that transform two-class membership values into membership probabilities. A common approach is to apply the logistic regression approach by Platt (1999). Zadrozny and Elkan (2002) supply an alternative method by using isotonic regression.
Multivariate extensions for regularization methods, i.e. number of classes greater than 2, can use a reduction to binary tasks, followed by univariate calibration with an algorithm as described above and further application of the pairwise coupling algorithm by Hastie and Tibshirani (1998).
An alternative one-step method, the Dirichlet calibration, is introduced by Gebel and Weihs (2008).
- Platt, J. C. (1999), "Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods", in Smola, Alexander J.; Bartlett, Peter J.; Schölkopf, Bernhard et al., Advances in Large Margin Classifiers, Cambridge: MIT Press, pp. 61–74, ISBN 978-0-262-19448-8, CiteSeerX: 10.1.1.41.1639
- Zadrozny, Bianca; Elkan, Charles (2002). "Transforming classifier scores into accurate multiclass probability estimates". Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '02. pp. 694–699. doi:10.1145/775047.775151. ISBN 1-58113-567-X. CiteSeerX: 10.1.1.13.7457.
- Hastie, Trevor; Tibshirani, Robert (1998). "Classification by pairwise coupling". The Annals of Statistics 26 (2): 451–471. doi:10.1214/aos/1028144844. Zbl 0932.62071. CiteSeerX: 10.1.1.46.6032.
- Gebel, Martin; Weihs, Claus (2008). "Calibrating Margin-Based Classifier Scores into Polychotomous Probabilities". In Preisach, Christine; Burkhardt, Hans; Schmidt-Thieme, Lars; Decker, Reinhold. Data Analysis, Machine Learning and Applications. Studies in Classification, Data Analysis, and Knowledge Organization. pp. 29–36. doi:10.1007/978-3-540-78246-9_4. ISBN 978-3-540-78239-1.