Type III error

In statistical hypothesis testing, there are various notions of so-called type III errors (or errors of the third kind), and sometimes type IV errors or higher, by analogy with the type I and type II errors of Jerzy Neyman and Egon Pearson. Fundamentally, Type III errors occur when researchers provide the right answer to the wrong question.

Since the paired notions of type I errors (or "false positives") and type II errors (or "false negatives") that were introduced by Neyman and Pearson are now widely used, their choice of terminology ("errors of the first kind" and "errors of the second kind"), has led others to suppose that certain sorts of mistakes that they have identified might be an "error of the third kind", "fourth kind", etc.

None of these proposed categories has been widely accepted. The following is a brief account of some of these proposals.

Systems theory

In systems theory an additional type III error is often defined: type III (δ): asking the wrong question and using the wrong null hypothesis.

David

Florence Nightingale David (1909–1993) [1] a sometime colleague of both Neyman and Pearson at the University College London, making a humorous aside at the end of her 1947 paper, suggested that, in the case of her own research, perhaps Neyman and Pearson's "two sources of error" could be extended to a third:

I have been concerned here with trying to explain what I believe to be the basic ideas [of my "theory of the conditional power functions"], and to forestall possible criticism that I am falling into error (of the third kind) and am choosing the test falsely to suit the significance of the sample.

— 1947, p.339

Mosteller

In 1948, Frederick Mosteller (1916–2006) argued that a "third kind of error" was required to describe circumstances he had observed, namely:

• Type I error: "rejecting the null hypothesis when it is true".
• Type II error: "accepting the null hypothesis when it is false".
• Type III error: "correctly rejecting the null hypothesis for the wrong reason". (1948, p. 61)

Kaiser

According to Henry F. Kaiser (1927–1992), in his 1966 paper extended Mosteller's classification such that an error of the third kind entailed an incorrect decision of direction following a rejected two-tailed test of hypothesis. In his discussion (1966, pp. 162–163), Kaiser also speaks of α errors, β errors, and γ errors for type I, type II and type III errors respectively (C.O. Dellomos).

Kimball

In 1957, Allyn W. Kimball, a statistician with the Oak Ridge National Laboratory, proposed a different kind of error to stand beside "the first and second types of error in the theory of testing hypotheses". Kimball defined this new "error of the third kind" as being "the error committed by giving the right answer to the wrong problem" (1957, p. 134).

Mathematician Richard Hamming (1915–1998) expressed his view that "It is better to solve the right problem the wrong way than to solve the wrong problem the right way".

Harvard economist Howard Raiffa describes an occasion when he, too, "fell into the trap of working on the wrong problem" (1968, pp. 264–265).

Mitroff and Featheringham

In 1974, Ian Mitroff and Tom Featheringham extended Kimball's category, arguing that "one of the most important determinants of a problem's solution is how that problem has been represented or formulated in the first place".

They defined type III errors as either "the error ... of having solved the wrong problem ... when one should have solved the right problem" or "the error ... [of] choosing the wrong problem representation ... when one should have ... chosen the right problem representation" (1974), p. 383.

In the 2009 book Dirty rotten strategies by Ian I. Mitroff and Abraham Silvers described type III and type IV errors providing many examples of both developing good answers to the wrong questions (III) and deliberately selecting the wrong questions for intensive and skilled investigation (IV). Most of the examples have nothing to do with statistics, many being problems of public policy or business decisions.

Raiffa

In 1969, the Harvard economist Howard Raiffa jokingly suggested "a candidate for the error of the fourth kind: solving the right problem too late" (1968, p. 264).

Marascuilo and Levin

In 1970, L. A. Marascuilo and J. R. Levin proposed a "fourth kind of error" – a "type IV error" – which they defined in a Mosteller-like manner as being the mistake of "the incorrect interpretation of a correctly rejected hypothesis"; which, they suggested, was the equivalent of "a physician's correct diagnosis of an ailment followed by the prescription of a wrong medicine" (1970, p. 398).

Russell Ackoff

In 2006, as part of his "f-laws" Russell Ackoff made a distinction between errors of commission and omission, or, in organizational science jargon, mistakes of commission and omission. A mistake of commission is something that the organization should not have done; a mistake of omission is something that the organization should have done. Ackoff suggested that mistakes of omission are much more serious, because they cannot be corrected or retrieved. Ackoff proposed that accounting systems in the western world only take account of errors of commission. Finally Ackoff proposed that a manager only has to be concerned about doing something that should not have been done in organizations that look down on mistakes and in which only errors of commission are identified. The Ackoff reference is important because it demonstrates applicability of the error typology in social sciences, as opposed to statistics, etc.

See also

• Howlers