Talk:Statistical hypothesis testing

From Wikipedia, the free encyclopedia
Jump to: navigation, search

The Cohen Criticism[edit]

I would like to expand on Cohen criticisms of NHST

*"[I]t does not tell us what we want to know"

by adding:

What we want to know is: given our data, what is the probability of the null hypothesis being true. But what the p-vlaue tells us is: given that the null hypothesis is true, what is the probability of obtaining our data? --1980na (talk) 01:23, 1 August 2014 (UTC)

It is perfectly possible to selectively quote Cohen as a Bayesian advocate. His recommendations in the subject article do not support that. He did not recommend replacing NHST, but supplementing it with exploratory data analysis and reporting effect sizes. (talk) 01:51, 7 August 2014 (UTC)

Grammatical error[edit]

As copied directly from the cited source here (potentially a copyright violation), the text under section /* The testing process */ reads: There should be a well-defined statistical statement (the null hypothesis (H0)) which allows to attach an attribute (rejected): it should be chosen in such a way that it allows us to conclude whether the alternative hypothesis can either be accepted or stays undecided as it was before the test. The grammatical error is the omission of the word "us" between "allows" and "to". But the larger problem is that, if I'm not mistaken, this is talking about setting the p-value, which is the probability of observing an effect given that the null hypothesis is true, but attempts to address the issue in a shorthand, that is less than clear outside of the context of the original source. --Bejnar (talk) 16:43, 12 August 2014 (UTC)

Well spotted. I removed that passage, and another from the same source, that were copied (or very closely paraphrased) from the book you linked to. They were added in this edit by an editor who stopped editing in 2010. Qwfp (talk) 17:32, 12 August 2014 (UTC)

Statistical threshold[edit]

On 17 August 2014, after discussion at Afd, Statistical threshold was redirected here, to "statistical hypothesis testing". --Bejnar (talk) 13:40, 17 August 2014 (UTC)

Origins and early controversy[edit]

Most papers on this subject say that textbooks are the source of the "hybrid" approach which is exemplified by the Lindquist figure in this wiki page. Yet the source that is cited in support of this (Halpin & Starn, 2006) say that textbooks are not the likely origin of the hybrid approach. To cite from page 644:

Thus, there seems to be little trace of a hybrid approach to statistical testing in the psychological research literature reviewed here, as indexed by the general omission of Neyman-Pearson concepts and procedures. Nonetheless, it has been found repeatedly that small-sample methods became prevalent in psychological research of this period. This suggests that a hybridized model of statistical testing was not directly transmitted from textbooks of psychological statistics to the research literature.

Clearly this Halpin & Starn is not valid to support this point. However, I am not knowledgeable enough about this the origins of the current Null Hypothesis Significance Testing procedure to say whether Halpin & Starn is right or not in the big picture. There is a big literature on this. Huberty (1993) "Historical origins of statistical testing practices: The treatment of Fisher versus Neyman-Pearson views in textbooks" is very relevant. — Preceding unsigned comment added by (talk) 10:24, 12 February 2015 (UTC)

I am not sure you read the entire paper. Halpin and Stam (2006) clearly do come to the conclusion the "hybrid" originated textbooks (the Lindquist figure claims to present evidence for this). The sentence following your quote begins the following:
This in turn raises the question of how to conceptualize the implications of the textbook hybrid for psychological research...
The historical research presented in this article therefore corroborates the interpretation of statistical textbooks given by Gigerenzer and Murray...
However, there has also been a conspicuous absence of Fisher's logic of inductive inference throughout the critical literature, an omission that has clear precedent in the hybrid model found in textbooks of psychological statistics.[1]
Of course whether this is the correct path the "idea" traveled along is another issue. There could be multiple points of origination. Huberty (1993) looks like it would make a great additional reference for this page. (talk) 21:50, 14 February 2015 (UTC)