False positives and false negatives: Difference between revisions
→False positive error: adds a citation for the definition of a false positive Tag: Reverted |
Simongraham (talk | contribs) Ref Tag: Reverted |
||
| Line 11: | Line 11: | ||
==False negative error== |
==False negative error== |
||
A '''false negative error''', or '''false negative''', is a test result which wrongly indicates that a condition does not hold. For example, when a pregnancy test indicates a woman is not pregnant, but she is, or when a person guilty of a crime is acquitted, these are false negatives. The condition "the woman is pregnant", or "the person is guilty" holds, but the test (the pregnancy test or the trial in a court of law) fails to realize this condition, and wrongly decides that the person is not pregnant or not guilty.{{ |
A '''false negative error''', or '''false negative''', is a test result which wrongly indicates that a condition does not hold. For example, when a pregnancy test indicates a woman is not pregnant, but she is, or when a person guilty of a crime is acquitted, these are false negatives. The condition "the woman is pregnant", or "the person is guilty" holds, but the test (the pregnancy test or the trial in a court of law) fails to realize this condition, and wrongly decides that the person is not pregnant or not guilty.<ref>{{cite book|last1=Alnabulsi|first1=Hussein|last2=Islam|first2=Rafiqui|last3=Mamun|first3=Qasi|year=2018|chapter=A novel algorithm to protect code injection attacks|editor-first1=Jemal H.|editor-last1=Abawajy|editor-first2=Kim-Kwang Raymond|editor-last2=Choo|editor-first3=Rafiqul|editor-last3=Islam| title=International Conference on Applications and Techniques in Cyber Security and Intelligence: Applications and Techniques in Cyber Security and Intelligence|publisher=Springer International Publishing|location=Cham, Switzerland|isbn=978-3-31967-071-3|page=288}}</ref> |
||
A false negative error is a [[type II error]] occurring in a test where a single condition is checked for, and the result of the test is erroneous, that the condition is absent.<ref name="fn">{{cite journal | title=Hypothesis testing, type I and type II errors | date=2009 | pmc=2996198 | pmid=21180491 | doi=10.4103/0972-6748.62274 | volume=18 | issue=2 | journal=Ind Psychiatry J | pages=127–31 | last1 = Banerjee | first1 = A | last2 = Chitnis | first2 = UB | last3 = Jadhav | first3 = SL | last4 = Bhawalkar | first4 = JS | last5 = Chaudhury | first5 = S | doi-access=free }}</ref> |
A false negative error is a [[type II error]] occurring in a test where a single condition is checked for, and the result of the test is erroneous, that the condition is absent.<ref name="fn">{{cite journal | title=Hypothesis testing, type I and type II errors | date=2009 | pmc=2996198 | pmid=21180491 | doi=10.4103/0972-6748.62274 | volume=18 | issue=2 | journal=Ind Psychiatry J | pages=127–31 | last1 = Banerjee | first1 = A | last2 = Chitnis | first2 = UB | last3 = Jadhav | first3 = SL | last4 = Bhawalkar | first4 = JS | last5 = Chaudhury | first5 = S | doi-access=free }}</ref> |
||
| Line 20: | Line 20: | ||
{{Main|Sensitivity and specificity|False positive rate}} |
{{Main|Sensitivity and specificity|False positive rate}} |
||
The '''[[false positive rate]]''' (FPR) is the proportion of all negatives that still yield positive test outcomes, i.e., the conditional probability of a positive test result given an event that was not present.{{ |
The '''[[false positive rate]]''' (FPR) is the proportion of all negatives that still yield positive test outcomes, i.e., the conditional probability of a positive test result given an event that was not present.<ref>{{cite journal|first1=Prosenjit|last1=Bose|first2=Hua|last2=Guo|first3=Evangelos|last3=Kranakis|first4=Anil|last4=Maheshwari|first5=Pat|last5=Morin|first6=Jason|last6=Morrison|first7=Michiel|last7=Smid|first8=Yihui|last8=Tang|title=On the false-positive rate of Bloom filters|journal=Information Processing Letters|volume=108|number=4|year=2008|page=210|doi=10.1016/j.ipl.2008.05.018}}</ref> The false positive rate depends on the [[significance level]]. The [[specificity (tests)|specificity]] of the test is equal to '''1''' minus the false positive rate.<ref>{{cite journal|first1=Paul|last1=Cronin|first2=Aine Marie|last2=Kelly|title=Influence of population prevalences on numbers of false positives: an overlooked entity|journal=Academic Radiology|volume=18|number=9|year=2011|page=1088|doi=10.1016/j.acra.2011.04.011}}</ref> |
||
The false positive rate is equal to the [[significance level]]. The [[specificity (tests)|specificity]] of the test is equal to '''1''' minus the false positive rate. |
|||
In [[statistical hypothesis testing]], this fraction is given the Greek letter ''α'', and 1 − ''α'' is defined as the specificity of the test. Increasing the specificity of the test lowers the probability of type I errors, but may raise the probability of type II errors (false negatives that reject the alternative hypothesis when it is true).{{efn|When developing detection algorithms or tests, a balance must be chosen between risks of false negatives and false positives. Usually there is a threshold of how close a match to a given sample must be achieved before the algorithm reports a match. The higher this threshold, the more false negatives and the fewer false positives.}} |
In [[statistical hypothesis testing]], this fraction is given the Greek letter ''α'', and 1 − ''α'' is defined as the specificity of the test. Increasing the specificity of the test lowers the probability of type I errors, but may raise the probability of type II errors (false negatives that reject the alternative hypothesis when it is true).{{efn|When developing detection algorithms or tests, a balance must be chosen between risks of false negatives and false positives. Usually there is a threshold of how close a match to a given sample must be achieved before the algorithm reports a match. The higher this threshold, the more false negatives and the fewer false positives.}} |
||
Complementarily, the '''{{visible anchor|false negative rate}}''' (FNR) is the proportion of positives which yield negative test outcomes with the test, i.e., the conditional probability of a negative test result given that the condition being looked for is present. |
Complementarily, the '''{{visible anchor|false negative rate}}''' (FNR) is the proportion of positives which yield negative test outcomes with the test, i.e., the conditional probability of a negative test result given that the condition being looked for is present.<ref>Cronin & Kelly, 2011, p.1087</ref> |
||
In [[statistical hypothesis testing]], this fraction is given the letter ''β''. The "[[statistical power|power]]" (or the "[[Sensitivity and specificity|sensitivity]]") of the test is equal to 1 − ''β''. |
In [[statistical hypothesis testing]], this fraction is given the letter ''β''. The "[[statistical power|power]]" (or the "[[Sensitivity and specificity|sensitivity]]") of the test is equal to 1 − ''β''. |
||
Revision as of 06:26, 9 June 2024
A false positive is an error in binary classification in which a test result incorrectly indicates the presence of a condition (such as a disease when the disease is not present), while a false negative is the opposite error, where the test result incorrectly indicates the absence of a condition when it is actually present. These are the two kinds of errors in a binary test, in contrast to the two kinds of correct result (a true positive and a true negative). They are also known in medicine as a false positive (or false negative) diagnosis, and in statistical classification as a false positive (or false negative) error.[1]
In statistical hypothesis testing, the analogous concepts are known as type I and type II errors, where a positive result corresponds to rejecting the null hypothesis, and a negative result corresponds to not rejecting the null hypothesis. The terms are often used interchangeably, but there are differences in detail and interpretation due to the differences between medical testing and statistical hypothesis testing.
False positive error
A false positive error, or false positive, is a result that indicates a given condition exists when it does not. For example, a pregnancy test which indicates a woman is pregnant when she is not, or the conviction of an innocent person.[2]
A false positive error is a type I error where the test is checking a single condition, and wrongly gives an affirmative (positive) decision. However it is important to distinguish between the type 1 error rate and the probability of a positive result being false. The latter is known as the false positive risk (see Ambiguity in the definition of false positive rate, below).[3]
False negative error
A false negative error, or false negative, is a test result which wrongly indicates that a condition does not hold. For example, when a pregnancy test indicates a woman is not pregnant, but she is, or when a person guilty of a crime is acquitted, these are false negatives. The condition "the woman is pregnant", or "the person is guilty" holds, but the test (the pregnancy test or the trial in a court of law) fails to realize this condition, and wrongly decides that the person is not pregnant or not guilty.[4]
A false negative error is a type II error occurring in a test where a single condition is checked for, and the result of the test is erroneous, that the condition is absent.[5]
Related terms
False positive and false negative rates
The false positive rate (FPR) is the proportion of all negatives that still yield positive test outcomes, i.e., the conditional probability of a positive test result given an event that was not present.[6] The false positive rate depends on the significance level. The specificity of the test is equal to 1 minus the false positive rate.[7]
In statistical hypothesis testing, this fraction is given the Greek letter α, and 1 − α is defined as the specificity of the test. Increasing the specificity of the test lowers the probability of type I errors, but may raise the probability of type II errors (false negatives that reject the alternative hypothesis when it is true).[a]
Complementarily, the false negative rate (FNR) is the proportion of positives which yield negative test outcomes with the test, i.e., the conditional probability of a negative test result given that the condition being looked for is present.[8]
In statistical hypothesis testing, this fraction is given the letter β. The "power" (or the "sensitivity") of the test is equal to 1 − β.
Ambiguity in the definition of false positive rate
The term false discovery rate (FDR) was used by Colquhoun (2014)[9] to mean the probability that a "significant" result was a false positive. Later Colquhoun (2017)[3] used the term false positive risk (FPR) for the same quantity, to avoid confusion with the term FDR as used by people who work on multiple comparisons. Corrections for multiple comparisons aim only to correct the type I error rate, so the result is a (corrected) p-value. Thus they are susceptible to the same misinterpretation as any other p-value. The false positive risk is always higher, often much higher, than the p-value.[9][3]
Confusion of these two ideas, the error of the transposed conditional, has caused much mischief.[10] Because of the ambiguity of notation in this field, it is essential to look at the definition in every paper. The hazards of reliance on p-values was emphasized in Colquhoun (2017)[3] by pointing out that even an observation of p = 0.001 was not necessarily strong evidence against the null hypothesis. Despite the fact that the likelihood ratio in favor of the alternative hypothesis over the null is close to 100, if the hypothesis was implausible, with a prior probability of a real effect being 0.1, even the observation of p = 0.001 would have a false positive rate of 8 percent. It wouldn't even reach the 5 percent level. As a consequence, it has been recommended[3][11] that every p-value should be accompanied by the prior probability of there being a real effect that it would be necessary to assume in order to achieve a false positive risk of 5%. For example, if we observe p = 0.05 in a single experiment, we would have to be 87% certain that there was a real effect before the experiment was done to achieve a false positive risk of 5%.
Receiver operating characteristic
The article "Receiver operating characteristic" discusses parameters in statistical signal processing based on ratios of errors of various types.
See also
- Base rate fallacy
- False positive rate
- Positive and negative predictive values
- Why Most Published Research Findings Are False
Notes
- ^ When developing detection algorithms or tests, a balance must be chosen between risks of false negatives and false positives. Usually there is a threshold of how close a match to a given sample must be achieved before the algorithm reports a match. The higher this threshold, the more false negatives and the fewer false positives.
References
- ^ False Positives and False Negatives
- ^ Robinson A, Keller LR, del Campo C. Building insights on true positives vs. false positives: Bayes’ rule. Decision Sciences Journal of Innovative Education. 2022;20(4):224-234. doi:10.1111/dsji.12265
- ^ a b c d e Colquhoun, David (2017). "The reproducibility of research and the misinterpretation of p-values". Royal Society Open Science. 4 (12): 171085. doi:10.1098/rsos.171085. PMC 5750014. PMID 29308247.
- ^ Alnabulsi, Hussein; Islam, Rafiqui; Mamun, Qasi (2018). "A novel algorithm to protect code injection attacks". In Abawajy, Jemal H.; Choo, Kim-Kwang Raymond; Islam, Rafiqul (eds.). International Conference on Applications and Techniques in Cyber Security and Intelligence: Applications and Techniques in Cyber Security and Intelligence. Cham, Switzerland: Springer International Publishing. p. 288. ISBN 978-3-31967-071-3.
- ^ Banerjee, A; Chitnis, UB; Jadhav, SL; Bhawalkar, JS; Chaudhury, S (2009). "Hypothesis testing, type I and type II errors". Ind Psychiatry J. 18 (2): 127–31. doi:10.4103/0972-6748.62274. PMC 2996198. PMID 21180491.
- ^ Bose, Prosenjit; Guo, Hua; Kranakis, Evangelos; Maheshwari, Anil; Morin, Pat; Morrison, Jason; Smid, Michiel; Tang, Yihui (2008). "On the false-positive rate of Bloom filters". Information Processing Letters. 108 (4): 210. doi:10.1016/j.ipl.2008.05.018.
- ^ Cronin, Paul; Kelly, Aine Marie (2011). "Influence of population prevalences on numbers of false positives: an overlooked entity". Academic Radiology. 18 (9): 1088. doi:10.1016/j.acra.2011.04.011.
- ^ Cronin & Kelly, 2011, p.1087
- ^ a b Colquhoun, David (2014). "An investigation of the false discovery rate and the misinterpretation of p-values". Royal Society Open Science. 1 (3): 140216. arXiv:1407.5296. Bibcode:2014RSOS....140216C. doi:10.1098/rsos.140216. PMC 4448847. PMID 26064558.
- ^ Colquhoun, David. "The problem with p-values". Aeon. Aeon Magazine. Retrieved 11 December 2016.
- ^ Colquhoun, David (2018). "The false positive risk: A proposal concerning what to do about p values". The American Statistician. 73: 192–201. arXiv:1802.04888. doi:10.1080/00031305.2018.1529622. S2CID 85530643.