Illusion of validity
Here is an imaginary example of illusion of validity. Let us picture a large hypothetical corporation. It only recruits people of a certain kind. Only those are wanted who have a prestigious college education which requires high SAT-scores. Also, job candidates are forced to participate in IQ tests. This despite the fact that some studies have shown a high degree of correlation between IQ and SAT scores, so once you know someone's SAT score, knowing their IQ in addition would not add much information and should increase confidence only very little; and, to whatever degree SAT scores correlate with GPA scores, GPA could be predicted from an IQ score nearly as effectively as SAT score.
Kahneman describes this bias as the first cognitive bias he conceived, in which he evaluated officer candidates for the Israeli Defense Forces according to a particular test. While serving in the Israeli army as a psychologist at 21 years old, he conducted a test to select candidates for officer training called the Leaderless Group Test. Eight participants were stripped of all identifying marks of rank and insignia, and set about navigating an obstacle course while meeting certain conditions. In particular, the participants were to maneuver a telephone pole through the course as well as themselves; one of the obstacles was a large vertical wall which both candidates and pole must pass over, without either ever touching the wall or the pole touching the ground. The candidates were allowed to organize themselves in whatever fashion they saw fit in order to complete this test. Kahneman and his peers observed and submitted notes to their superiors, and periodically received statistical information validating the correctness of their evaluations. While the observers consistently received poor marks in predicting a candidate's performance in officer training school, the team incongruously remained optimistic in their ability to correctly predict future outcomes.
Kahnemann on YouTube: Explorations of the Mind: Intuition
- Meinolf Dierkes; Ariane Berthoin Antal; John Child; Ikujiro Nonaka (2003). Handbook of Organizational Learning and Knowledge. Oxford University Press. p. 22. ISBN 978-0-19-829582-2. Retrieved 9 September 2013. http://books.google.se/books?id=JRd7RZzzw_wC&pg=PA22&dq=Illusion+of+validity&hl=en&sa=X&ei=lr4tUoKtOKKK4wSi3IEw&redir_esc=y#v=onepage&q=Illusion%20of%20validity&f=false
- Frey, Meredith C.; Douglas K. Detterman (2004). "Scholastic Assessment org?". Psychological Science 15 (6): 373–378. doi:10.1111/j.0956-7976.2004.00687.x. PMID 15147489.
- Einhorn, Hillel; Robyn M. Hogarth (1978). "Confidence in judgment: Persistence of the illusion of validity". Psychological Review 85 (5): 395–416. doi:10.1037/0033-295x.85.5.395.
- Einhorn & Hogarth, 1978
- "'Thinking, Fast and Slow'". 'KQED Public Media for Northern California'.