Expert elicitation

From Wikipedia, the free encyclopedia
Jump to: navigation, search

In science, engineering, and research, expert elicitation is the synthesis of opinions of authorities of a subject where there is uncertainty due to insufficient data or when such data is unattainable because of physical constraints or lack of resources.[1] Expert elicitation is essentially a scientific consensus methodology. It is often used in the study of rare events. Expert elicitation allows for parametrization, an "educated guess", for the respective topic under study. Expert elicitation generally quantifies uncertainty.

Expert elicitation tends to be multidisciplinary as well as interdisciplinary, with practically universal applicability, and is used in a broad range of fields. Prominent recent expert elicitation applications include climate change, modeling seismic hazard and damage, association of tornado damage to wind speed in developing the Enhanced Fujita scale, risk analysis for nuclear waste storage.

In performing expert elicitation certain factors need to be taken into consideration. The topic must be one for which there are people who have predictive expertise. Furthermore, the objective should be to obtain an experts' carefully considered judgment based on a systematic consideration of all relevant evidence. For this reason one should take care to adopt strategies designed to help the expert being interviewed to avoid overlooking relevant evidence. Additionally, vocabulary used should face intense scrutiny; qualitative uncertainty words such as "likely" and "unlikely" are not sufficient and can lead to confusion. Such words can mean very different things to different people, or to the same people in different situations.[2]

To increase the reliability in expert judgments, the literature provides criteria and procedures for eliciting this information while reducing the risk of obtaining biased judgments (e.g., Cooke and Goossens,[3] Ayyub,[4] Garthwaite et al.,[5] Goossens et al.,[6] Kynn,[7] Hallowell and Gambatese,[8] and Cardenas et al.[9]

See also[edit]


  1. ^ van der Sluijs, et. al., Jeroen P. (2008). "Expert Elicitation: Methodological suggestions for its use in environmental health impact assessments" (PDF). NUSAP. Retrieved 25 November 2015. 
  2. ^ Tversky, Amos; Kahneman, Daniel (27 September 1974). Judgments under uncertainty: Heuristics and biases (PDF) (Vol. 185, No. 4157 ed.). Science. pp. 1124–1131. Retrieved 25 November 2015. 
  3. ^ Cooke, RM; Goossens, LHJ (2000). "Procedures guide for structural expert judgement in accident consequence modelling". Radiat Prot Dosimetry. 90 (3). 
  4. ^ Ayyub, BM (2001). Elicitation of Expert Opinions for Uncertainty and Risks (1st ed ed.). CRC Press. pp. 328 p. 
  5. ^ Garthwaite, PH; Kadane, JB; O’Hagan, A (2005). "Statistical methods for eliciting probability distributions". J Am Stat Assoc. 100 (470). 
  6. ^ Goossens, LHJ; Cooke, RM; Hale, AR; Rodic-Wiersma, LJ (2008). "Fifteen years of expert judgement at TUDelft". Safety Sci. 46 (2). 
  7. ^ Kynn, M (2008). "The ‘heuristics and biases’ bias in expert elicitation". J R Stat Soc A. 171 (1). 
  8. ^ Hallowell, MR; Gambatese, JA (2010). "Qualitative research: Application of the Delphi method to CEM research". J Constr Eng Manage. 136 (1). 
  9. ^ Cárdenas, IC; Al-Jibouri, SHS; Halman, JIM; van Tol, FA (2013). "Capturing and Integrating Knowledge for Managing Risks in Tunnel Works". Risk Analysis. 33 (1): 92–108. doi:10.1111/j.1539-6924.2012.01829.x. 


  • Apostolakis, G., 7 December 1990: The concept of probability in safety assessments of technological systems. Science, 250 (4986): 1359-1364. doi:10.1126/science.2255906
  • Arkes, Hal R., Jeryl L. Mumpower, and Thomas R. Stewart, 24 January 1997: Combining Expert Opinions. Science, 275: 461-465. doi:10.1126/science.275.5299.461e
  • Boissonnade, A., Hossain, Q., Kimbell, J., Mensing, R., and Savy, J., 2000: Development of a probabilistic tornado wind hazard model for the Continental United States, UCRL-ID-140922 Vol. I, Lawrence Livermore National Laboratory, Livermore, CA, 131pp.
  • Kerr, Richard A., 8 November 1996: Risk Assessment: A New Way to Ask the Experts: Rating Radioactive Waste Risks. Science, 274 (5289): 913-914. doi:10.1126/science.274.5289.913
  • SSHAC, 1997: Recommendations for probabilistic seismic hazard analysis: guidelines on uncertainty and use of experts, NUREG/CR-6372, UCRL-ID-122160, Vol. I, Lawrence Livermore National Laboratory, Livermore, CA, 131 pp.