Overconfidence effect

From Wikipedia, the free encyclopedia

Jump to: navigation, search

The overconfidence effect is a well-established bias in which someone's subjective confidence in their judgements is reliably greater than their objective accuracy, especially when confidence is relatively high.[1] For example, in some quizzes, people rate their answers as "99% certain" but are wrong 40% of the time. It has been proposed[2] that a metacognitive trait mediates the accuracy of confidence judgments, but this trait's relationship to variations in cognitive ability and personality remains uncertain.[1] Overconfidence is one example of a miscalibration of subjective probabilities.

Contents

[edit] Demonstration

The effect has been demonstrated in a number of different ways.[3] Among them are:

  1. Request subjects to evaluate their confidence in a statement. Group together all the statements with a given level of confidence (e.g., 90%) and compare that to the actual frequency of being correct.
  2. Test subjects with multiple-choice questions and then elicit their level of confidence in their answer on a scale from chance to 100% (total certainty). Compare this to the true accuracy of the answers.
  3. Give subjects a question with a numerical answer, and get them to choose a confidence interval such that they have have a particular level of confidence that the true answer is in that range. E.g., "Pick a low number and a high number such that you are 90% confident that the population of Bulgaria is between those numbers."
  4. Offer subjects the opportunity to bet on the correctness of their answers with chances that are favorable, if their judgements of accuracy are correct. They lose money if they are overconfident.[4]

If human confidence had perfect calibration, judgements with 100% confidence would be correct 100% of the time, 90% confidence correct 90% of the time, and so on for the other levels of confidence. By contrast, the key finding is that confidence exceeds accuracy so long as the subject is answering hard questions about an unfamiliar topic. In a spelling task, subjects were correct about 80% of the time when they were "100% certain".[5] Put another way, the error rate was 20% when subjects expected it to be 0%. In a series where subjects made true-or-false responses to general knowledge statements, they were overconfident at all levels. When they were 100% certain of their answer to a question, they were wrong 20% of the time. [6] A In a confidence-intervals task, where subjects had to judge quantities such as the total egg production of the U.S. or the total number of physicians and surgeons in the Boston Yellow Pages, they expected an error rate of 2% when their real error rate was 46%. Even after subjects had been thoroughly warned about the bias, they still showed a high degree of overconfidence.[7]

Different classes of experts have been tested for overconfidence within their area of expertise, with varying results. A test of 25,000 predictions by weather forecasters found no overconfidence, but an experiment involving clinical psychologists found a high margin of overconfidence in their conclusions about a specific case (see below).

When the accuracy is close to 100%, confidence is usually somewhat less, that is, underconfidence. This is interpretable as a ceiling effect (i.e., at the top of the scale, errors can only be made in one direction).

[edit] In specific areas

Stuart Oskamp[8] tested groups of clinical psychologists and psychology students on a multiple-choice task in which they drew conclusions from a case study. Along with their answers, subjects gave a confidence rating in the form of a percentage likelihood of being correct. This allowed confidence to be compared against accuracy. As the subjects were given more information about the case study, their confidence increased from 33% to 53%. However, their accuracy did not significantly improve, staying under 30%. Hence, this experiment demonstrated overconfidence which increased as the subjects had more information on which to base their judgment.

[edit] Related biases

[edit] Optimism bias

Main article: Optimism bias

In a "winner-takes-all" context, many real-world situations can be characterized as a rank order tournament. In other words, prizes are not proportional to outcomes, but accrue to the top performers. In many employment situations, only the best performers are promoted (e.g., tenure in academia or promotion to partner in consulting or law firms). In such situations, overconfidence bias may:

  1. Cause employees to overestimate their chances of promotion, this causes them to overly value the possible promotion as an incentive, because they deem it to be more likely to occur.
  2. Cause employees to work harder than they normally would for a chance at a promotion.
  3. Prefer tournaments and other compensation schemes where most of the rewards are concentrated at the top, and where the costs of failure are extreme (for example an "up or out" promotion system). The overconfidence bias causes these employees to consider the chance of failure to be very slim and to overestimate the chance of success.

[edit] Escalating commitment

Overconfidence bias often serves to increase the effects of escalating commitment[citation needed]—causing decision makers to refuse to withdraw from a losing situation, or to continue to throw good money, effort, time and other resources after bad investments.

[edit] Ignoring base rates

People often tend to ignore base rates (cf. base rate neglect) or undervalue their effect. For example, if one is competing against individuals who are already winners of previous competitions, one's odds of winning should be adjusted downward considerably. People tend to fail to do so sufficiently.[citation needed]

[edit] See also

Psychology portal

[edit] Notes

  1. ^ a b Pallier, Gerry, et al. "The role of individual differences in the accuracy of confidence judgments." The Journal of General Psychology 129.3 (2002): 257+.
  2. ^ Stankov, L. (1999). Mining on the "no man's land" between intelligence and personality. In P. L. Ackerman, P. C. Kyllonen, & R. D. Roberts (Eds.), Learning and individual differences: Process, trait, and content determinants (pp. 314-337). Washington, DC: American Psychological Association.
  3. ^ For surveys, see Hoffrage (2004) or Lichtenstein et al. (1982))
  4. ^ Fischoff, Baruch; Paul Slovic, Sarah Lichtenstein (1977). "Knowing with certainty: The appropriateness of extreme confidence". Journal of Experimental Psychology: Human Perception and Performance 3: 552–564. doi:10.1037/0096-1523.3.4.552. 
  5. ^ Adams, P. A. & Adams, J. K . (1960). Confidence in the recognition and reproduction of words difficult to spell. American Journal of Psychology, 73 pp. 544-552
  6. ^ Lichtenstein, Sarah; Baruch Fischoff, Lawrence D. Phillips (1982). "Calibration of probabilities: The state of the art to 1980". in Daniel Kahneman, Paul Slovic, Amos Tversky. Judgment under uncertainty: Heuristics and biases. Cambridge University Press. pp. 306–334. ISBN 978-0-521-28414-1. 
  7. ^ Alpert, Marc; Howard Raiffa (1982). "A progress report on the training of probability assessors". in Daniel Kahneman, Paul Slovic, Amos Tversky. Judgment under uncertainty: Heuristics and biases. Cambridge University Press. pp. 294–305. ISBN 9780521284141. 
  8. ^ Oskamp, Stuart (1965). "Overconfidence in case-study judgements". The Journal of Consulting Psychology (American Psychological Association) 2: 261–265.  reprinted in Kahneman, Daniel; Paul Slovic, Amos Tversky (1982). Judgment under uncertainty: Heuristics and biases. Cambridge University Press. pp. 287–293. ISBN 978-0-521-28414-1. .

[edit] References

  • Baron, Johnathan (1994). Thinking and Deciding. Cambridge University Press. pp. 219–224. ISBN 0-521-43732-6. 
  • Gilovich, Thomas; Dale Griffin, Daniel Kahneman (Eds.). (2002). Heuristics and biases: The psychology of intuitive judgment. Cambridge, UK: Cambridge University Press. ISBN 0-521-79679-2
  • Hoffrage, Ulrich (2004). "Overconfidence". in Rüdiger Pohl. Cognitive Illusions: a handbook on fallacies and biases in thinking, judgement and memory. Psychology Press. ISBN 978-1-84169-351-4. 
  • Shefrin, H. (2002). Beyond Greed and Fear: Understanding Behavioral Finance and the Psychology of Investing. Oxford University Press.
  • Sutherland, Stuart (2007). Irrationality. Pinter & Martin. pp. 172–178. ISBN 978-1-905177-07-3. 

[edit] External links

Retrieved from "http://en.wikipedia.org/wiki/Overconfidence_effect"
Languages