Probability matching

Probability matching is a decision strategy in which predictions of class membership are proportional to the class base rates. Thus, if in the training set positive examples are observed 60% of the time, and negative examples are observed 40% of the time, then the observer using a probability-matching strategy will predict (for unlabeled examples) a class label of "positive" on 60% of instances, and a class label of "negative" on 40% of instances.

The optimal Bayesian decision strategy (to maximize the number of correct predictions, see Duda, Hart & Stork (2001)) in such a case is to always predict "positive" (i.e., predict the majority category in the absence of other information), which has 60% chance of winning rather than matching which has 52% of winning (where p is the probability of positive realization, the result of matching would be $p^{2}+(1-p)^{2}$, here $.6\times .6+.4\times .4$). The probability-matching strategy is of psychological interest because it is frequently employed by human subjects in decision and classification studies (where it may be related to Thompson sampling).

References

• Duda, Richard O.; Hart, Peter E.; Stork, David G. (2001), Pattern Classification (2nd ed.), New York: John Wiley & Sons

• Shanks, D. R., Tunney, R. J., & McCarthy, J. D. (2002). A re‐examination of probability matching and rational choice. Journal of Behavioral Decision Making, 15(3), 233-250.