In metrology, measurement uncertainty is a non-negative parameter characterizing the dispersion of the values attributed to a measured quantity. The uncertainty has a probabilistic basis and reflects incomplete knowledge of the quantity. All measurements are subject to uncertainty and a measured value is only complete if it is accompanied by a statement of the associated uncertainty. Relative uncertainty is the measurement uncertainty divided by the measured value.
- 1 Background
- 2 Random and systematic errors
- 3 Measurement model
- 4 Propagation of distributions
- 5 Type A and Type B evaluation of uncertainty
- 6 Sensitivity coefficients
- 7 Uncertainty evaluation
- 8 Alternative perspective
- 9 See also
- 10 References
- 11 Further reading
The purpose of measurement is to provide information about a quantity of interest – a measurand. For example, the measurand might be the size of a cylindrical feature per ASME Y14.5-2009, the volume of a vessel, the potential difference between the terminals of a battery, or the mass concentration of lead in a flask of water.
No measurement is exact. When a quantity is measured, the outcome depends on the measuring system, the measurement procedure, the skill of the operator, the environment, and other effects. Even if the quantity were to be measured several times, in the same way and in the same circumstances, a different measured value would in general be obtained each time, assuming the measuring system has sufficient resolution to distinguish between the values.
The dispersion of the measured values would relate to how well the measurement is made. Their average would provide an estimate of the true value of the quantity that generally would be more reliable than an individual measured value. The dispersion and the number of measured values would provide information relating to the average value as an estimate of the true value. However, this information would not generally be adequate.
The measuring system may provide measured values that are not dispersed about the true value, but about some value offset from it. Take a domestic bathroom scale. Suppose it is not set to show zero when there is nobody on the scale, but to show some value offset from zero. Then, no matter how many times the person's mass were re-measured, the effect of this offset would be inherently present in the average of the values.
Measurement uncertainty has important economic consequences for calibration and measurement activities. In calibration reports, the magnitude of the uncertainty is often taken as an indication of the quality of the laboratory, and smaller uncertainty values generally are of higher value and of higher cost. The American Society of Mechanical Engineers (ASME) has produced a suite of standards addressing various aspects of measurement uncertainty. ASME B126.96.36.199, Guidelines for Decision Rules in Determining Conformance to Specifications addresses the role of measurement uncertainty when accepting or rejecting products based on a measurement result and a product specification. ASME B188.8.131.52, Guidelines for the Evaluation of Dimensional Measurement Uncertainty, provides a simplified approach (relative to the GUM) to the evaluation of dimensional measurement uncertainty. ASME B184.108.40.206, Guidelines for Assessing the Reliability of Dimensional Measurement Uncertainty Statements, examines how to resolve disagreements over the magnitude of the measurement uncertainty statement. ASME B89.7.4, Measurement Uncertainty and Conformance Testing: Risk Analysis, provides guidance on the risks involved in any product acceptance/rejection decision.
The "Guide to the Expression of Uncertainty in Measurement", commonly known as the GUM, is the definitive document on this subject. The GUM has been adopted by all major National Measurement Institutes (NMIs), by international laboratory accreditation standards such as ISO 17025 which is required for ILAC accreditation, and employed in most modern national and international documentary standards on measurement methods and technology. See Joint Committee for Guides in Metrology.
Random and systematic errors
A systematic error (an estimate of which is known as a measurement bias) is associated with the fact that a measured value contains an offset. In general, a systematic error, regarded as a quantity, is a component of error that remains constant or depends in a specific manner on some other quantity.
A random error is associated with the fact that when a measurement is repeated it will generally provide a measured value that is different from the previous value. It is random in that the next measured value cannot be predicted exactly from previous such values. (If a prediction were possible, allowance for the effect could be made.)
In general, there can be a number of contributions to each type of error.
The Performance Test Standard PTC 19.1-2005 “Test Uncertainty”, published by ASME, discusses systematic and random errors in considerable detail. In fact, it conceptualizes its basic uncertainty categories in these terms.
The above discussion concerns the direct measurement of a quantity, which incidentally occurs rarely. For example, the bathroom scale may convert a measured extension of a spring into an estimate of the measurand, the mass of the person on the scale. The particular relationship between extension and mass is determined by the calibration of the scale. A measurement model converts a quantity value into the corresponding value of the measurand.
There are many types of measurement in practice and therefore many models. A simple measurement model (for example for a scale, where the mass is proportional to the extension of the spring) might be sufficient for everyday domestic use. Alternatively, a more sophisticated model of a weighing, involving additional effects such as air buoyancy, is capable of delivering better results for industrial or scientific purposes. In general there are often several different quantities, for example temperature, humidity and displacement, that contribute to the definition of the measurand, and that need to be measured.
Correction terms should be included in the measurement model when the conditions of measurement are not exactly as stipulated. These terms correspond to systematic errors. Given an estimate of a correction term, the relevant quantity should be corrected by this estimate. There will be an uncertainty associated with the estimate, even if the estimate is zero, as is often the case. Instances of systematic errors arise in height measurement, when the alignment of the measuring instrument is not perfectly vertical, and the ambient temperature is different from that prescribed. Neither the alignment of the instrument nor the ambient temperature is specified exactly, but information concerning these effects is available, for example the lack of alignment is at most 0.001° and the ambient temperature at the time of measurement differs from that stipulated by at most 2 °C.
As well as raw data representing measured values, there is another form of data that is frequently needed in a measurement model. Some such data relate to quantities representing physical constants, each of which is known imperfectly. Examples are material constants such as modulus of elasticity and specific heat. There are often other relevant data given in reference books, calibration certificates, etc., regarded as estimates of further quantities.
The items required by a measurement model to define a measurand are known as input quantities in a measurement model. The model is often referred to as a functional relationship. The output quantity in a measurement model is the measurand.
Formally, the output quantity, denoted by , about which information is required, is often related to input quantities, denoted by , about which information is available, by a measurement model in the form of
where is known as the measurement function. A general expression for a measurement model is
It is taken that a procedure exists for calculating given , and that is uniquely defined by this equation.
Propagation of distributions
The true values of the input quantities are unknown. In the GUM approach, are characterized by probability distributions and treated mathematically as random variables. These distributions describe the respective probabilities of their true values lying in different intervals, and are assigned based on available knowledge concerning . Sometimes, some or all of are interrelated and the relevant distributions, which are known as joint, apply to these quantities taken together.
Consider estimates , respectively, of the input quantities , obtained from certificates and reports, manufacturers' specifications, the analysis of measurement data, and so on. The probability distributions characterizing are chosen such that the estimates , respectively, are the expectations of . Moreover, for the th input quantity, consider a so-called standard uncertainty, given the symbol , defined as the standard deviation of the input quantity . This standard uncertainty is said to be associated with the (corresponding) estimate .
The use of available knowledge to establish a probability distribution to characterize each quantity of interest applies to the and also to . In the latter case, the characterizing probability distribution for is determined by the measurement model together with the probability distributions for the . The determination of the probability distribution for from this information is known as the propagation of distributions.
The figure below depicts a measurement model in the case where and are each characterized by a (different) rectangular, or uniform, probability distribution. has a symmetric trapezoidal probability distribution in this case.
Once the input quantities have been characterized by appropriate probability distributions, and the measurement model has been developed, the probability distribution for the measurand is fully specified in terms of this information. In particular, the expectation of is used as the estimate of , and the standard deviation of as the standard uncertainty associated with this estimate.
Often an interval containing with a specified probability is required. Such an interval, a coverage interval, can be deduced from the probability distribution for . The specified probability is known as the coverage probability. For a given coverage probability, there is more than one coverage interval. The probabilistically symmetric coverage interval is an interval for which the probabilities (summing to one minus the coverage probability) of a value to the left and the right of the interval are equal. The shortest coverage interval is an interval for which the length is least over all coverage intervals having the same coverage probability.
Prior knowledge about the true value of the output quantity can also be considered. For the domestic bathroom scale, the fact that the person's mass is positive, and that it is the mass of a person, rather than that of a motor car, that is being measured, both constitute prior knowledge about the possible values of the measurand in this example. Such additional information can be used to provide a probability distribution for that can give a smaller standard deviation for and hence a smaller standard uncertainty associated with the estimate of .
Type A and Type B evaluation of uncertainty
Knowledge about an input quantity is inferred from repeated measured values (Type A evaluation of uncertainty), or scientific judgement or other information concerning the possible values of the quantity (Type B evaluation of uncertainty).
In Type A evaluations of measurement uncertainty, the assumption is often made that the distribution best describing an input quantity given repeated measured values of it (obtained independently) is a Gaussian distribution. then has expectation equal to the average measured value and standard deviation equal to the standard deviation of the average. When the uncertainty is evaluated from a small number of measured values (regarded as instances of a quantity characterized by a Gaussian distribution), the corresponding distribution can be taken as a -distribution. Other considerations apply when the measured values are not obtained independently.
For a Type B evaluation of uncertainty, often the only available information is that lies in a specified interval . In such a case, knowledge of the quantity can be characterized by a rectangular probability distribution with limits and . If different information were available, a probability distribution consistent with that information would be used.
Sensitivity coefficients describe how the estimate of would be influenced by small changes in the estimates of the input quantities . For the measurement model , the sensitivity coefficient equals the partial derivative of first order of with respect to evaluated at , , etc. For a linear measurement model
with independent, a change in equal to would give a change in . This statement would generally be approximate for measurement models . The relative magnitudes of the terms are useful in assessing the respective contributions from the input quantities to the standard uncertainty associated with .
The standard uncertainty associated with the estimate of the output quantity is not given by the sum of the , but these terms combined in quadrature, namely by [an expression that is generally approximate for measurement models ]
which is known as the law of propagation of uncertainty.
The main stages of uncertainty evaluation constitute formulation and calculation, the latter consisting of propagation and summarizing. The formulation stage constitutes
- defining the output quantity (the measurand),
- identifying the input quantities on which depends,
- developing a measurement model relating to the input quantities, and
- on the basis of available knowledge, assigning probability distributions — Gaussian, rectangular, etc. — to the input quantities (or a joint probability distribution to those input quantities that are not independent).
The calculation stage consists of propagating the probability distributions for the input quantities through the measurement model to obtain the probability distribution for the output quantity , and summarizing by using this distribution to obtain
- the expectation of , taken as an estimate of ,
- the standard deviation of , taken as the standard uncertainty associated with , and
- a coverage interval containing with a specified coverage probability.
The propagation stage of uncertainty evaluation is known as the propagation of distributions, various approaches for which are available, including
- the GUM uncertainty framework, constituting the application of the law of propagation of uncertainty, and the characterization of the output quantity by a Gaussian or a -distribution,
- analytic methods, in which mathematical analysis is used to derive an algebraic form for the probability distribution for , and
- a Monte Carlo method, in which an approximation to the distribution function for is established numerically by making random draws from the probability distributions for the input quantities, and evaluating the model at the resulting values.
For any particular uncertainty evaluation problem, approach 1), 2) or 3) (or some other approach) is used, 1) being generally approximate, 2) exact, and 3) providing a solution with a numerical accuracy that can be controlled.
Models with any number of output quantities
When the measurement model is multivariate, that is, it has any number of output quantities, the above concepts can be extended. The output quantities are now described by a joint probability distribution, the coverage interval becomes a coverage region, the law of propagation of uncertainty has a natural generalization, and a calculation procedure that implements a multivariate Monte Carlo method is available.
Most of this article represents the most common view of measurement uncertainty, which assumes that random variables are proper mathematical models for uncertain quantities and simple probability distributions are sufficient for representing all forms of measurement uncertainties. In some situations, however, a mathematical interval rather than a probability distribution might be a better model of uncertainty. This may include situations involving periodic measurements, binned data values, censoring, detection limits, or plus-minus ranges of measurements where no particular probability distribution seems justified or where one cannot assume that the errors among individual measurements are completely independent.
A more robust representation of measurement uncertainty in such cases can be fashioned from intervals. An interval [a,b] is different from a rectangular or uniform probability distribution over the same range in that the latter suggests that the true value lies inside the right half of the range [(a + b)/2, b] with probability one half, and within any subinterval of [a,b] with probability equal to the width of the subinterval divided by b – a. The interval makes no such claims, except simply that the measurement lies somewhere within the interval. Distributions of such measurement intervals can be summarized as probability boxes and Dempster–Shafer structures over the real numbers, which incorporate both aleatoric and epistemic uncertainties.
- Bell, S. Measurement Good Practice Guide No. 11. A Beginner's Guide to Uncertainty of Measurement. Tech. rep., National Physical Laboratory, 1999.
- JCGM 101:2008. Evaluation of measurement data – Supplement 1 to the "Guide to the expression of uncertainty in measurement" – Propagation of distributions using a Monte Carlo method. Joint Committee for Guides in Metrology.
- Bernardo, J., and Smith, A. Bayesian Theory. John Wiley & Sons, New York, USA, 2000. 3.20
- Elster, C. Calculation of uncertainty in the presence of prior knowledge. Metrologia 44 (2007), 111–116. 3.20
- EURACHEM/CITAC. Quantifying uncertainty in analytical measurement. Tech. Rep. Guide CG4, EU-RACHEM/CITEC,[EURACHEM/CITAC Guide], 2000. Second edition.
- JCGM 104:2009. Evaluation of measurement data – An introduction to the "Guide to the expression of uncertainty in measurement" and related documents. Joint Committee for Guides in Metrology.
- Weise, K., and Wöger, W. A Bayesian theory of measurement uncertainty. Meas. Sci. Technol. 3 (1992), 1–11, 4.8.
- JCGM 100:2008. Evaluation of measurement data – Guide to the expression of uncertainty in measurement, Joint Committee for Guides in Metrology.
- Joint Committee for Guides in Metrology (2011). JCGM 102: Evaluation of Measurement Data – Supplement 2 to the "Guide to the Expression of Uncertainty in Measurement" – Extension to Any Number of Output Quantities (Technical report). JCGM. Retrieved 13 February 2013.
- Manski, C.F. (2003); Partial Identification of Probability Distributions, Springer Series in Statistics, Springer, New York
- Ferson, S., V. Kreinovich, J. Hajagos, W. Oberkampf, and L. Ginzburg (2007); Experimental Uncertainty Estimation and Statistics for Data Having Interval Uncertainty, Sandia National Laboratories SAND 2007-0939
- JCGM 200:2008. International Vocabulary of Metrology – Basic and general concepts and associated terms, 3rd Edition. Joint Committee for Guides in Metrology.
- ISO 3534-1:2006. Statistics – Vocabulary and symbols – Part 1: General statistical terms and terms used in probability. ISO
- JCGM 106:2012. Evaluation of measurement data – The role of measurement uncertainty in conformity assessment. Joint Committee for Guides in Metrology.
- Cox, M. G., and Harris, P. M. SSfM Best Practice Guide No. 6, Uncertainty evaluation. Technical report DEM-ES-011, National Physical Laboratory, 2006.
- Cox, M. G., and Harris, P. M . Software specifications for uncertainty evaluation. Technical report DEM-ES-010, National Physical Laboratory, 2006.
- Grabe, M ., Measurement Uncertainties in Science and Technology, Springer 2005.
- Grabe, M. Generalized Gaussian Error Calculus, Springer 2010.
- Dietrich, C. F. Uncertainty, Calibration and Probability. Adam Hilger, Bristol, UK, 1991.
- NIST. Uncertainty of measurement results.
- Bich, W., Cox, M. G., and Harris, P. M. Evolution of the "Guide to the Expression of Uncertainty in Measurement". Metrologia, 43(4):S161–S166, 2006.
- EA. Expression of the uncertainty of measurement in calibration. Technical Report EA-4/02, European Co-operation for Accreditation, 1999.
- Elster, C., and Toman, B. Bayesian uncertainty analysis under prior ignorance of the measurand versus analysis using Supplement 1 to the Guide: a comparison. Metrologia, 46:261–266, 2009.
- Ferson, S., Kreinovich, V., Hajagos, J., Oberkampf, W., and Ginzburg, L. 2007. "Experimental Uncertainty Estimation and Statistics for Data Having Interval Uncertainty". SAND2007-0939.
- Lira., I. Evaluating the Uncertainty of Measurement. Fundamentals and Practical Guidance. Institute of Physics, Bristol, UK, 2002.
- Majcen N., Taylor P. (Editors), Practical examples on traceability, measurement uncertainty and validation in chemistry, Vol 1, 2010; ISBN 978-92-79-12021-3.
- UKAS. The expression of uncertainty in EMC testing. Technical Report LAB34, United Kingdom Accreditation Service, 2002.
- UKAS M3003 The Expression of Uncertainty and Confidence in Measurement (Edition 3, November 2012) UKAS
- Estimate of temperature and its uncertainty in small systems, 2011.
- ASME PTC 19.1, Test Uncertainty, New York: The American Society of Mechanical Engineers; 2005
- Introduction to evaluating uncertainty of measurement
- Da Silva, R.B., Bulska, E., Godlewska-Zylkiewicz, B., Hedrich, M., Majcen, N., Magnusson, B., Marincic, S., Papadakis, I., Patriarca, M., Vassileva, E., Taylor, P., Analytical measurement: measurement uncertainty and statistics;ISBN 978-92-79-23070-7, 2012.
- Arnaut, L. R. Measurement uncertainty in reverberation chambers – I. Sample statistics. Technical report TQE 2 2nd. ed., National Physical Laboratory, 2008.
- Estimation of measurement uncertainty in chemical analysis (analytical chemistry) On-line course created by I. Leito, L. Jalukse and I. Helm, University of Tartu, 2013.