Cochran's Q test
In statistics, in the analysis of two-way randomized block designs where the response variable can take only two possible outcomes (coded as 0 and 1), Cochran's Q test is a non-parametric statistical test to verify whether k treatments have identical effects. It is named after William Gemmell Cochran. Cochran's Q test should not be confused with Cochran's C test, which is a variance outlier test. Put in simple technical terms, Cochran's Q test requires that there only be a binary response (e.g. success/failure or 1/0) and that there be more than 2 groups of the same size. The test assesses whether the proportion of successes is the same between groups. Often it is used to assess if different observers of the same phenomenon have consistent results (interobserver variability).
Cochran's Q test assumes that there are k > 2 experimental treatments and that the observations are arranged in b blocks; that is,
|Treatment 1||Treatment 2||Treatment k|
Cochran's Q test is
- Null hypothesis (H0): the treatments are equally effective.
- Alternative hypothesis (Ha): there is a difference in effectiveness between treatments.
The Cochran's Q test statistic is
- k is the number of treatments
- X• j is the column total for the jth treatment
- b is the number of blocks
- Xi • is the row total for the ith block
- N is the grand total
For significance level α, the asymptotic critical region is
where Χ21 − α,k − 1 is the (1 − α)-quantile of the chi-squared distribution with k − 1 degrees of freedom. The null hypothesis is rejected if the test statistic is in the critical region. If the Cochran test rejects the null hypothesis of equally effective treatments, pairwise multiple comparisons can be made by applying Cochran's Q test on the two treatments of interest.
The exact distribution of the T statistic may be computed for small samples. This allows obtaining an exact critical region. A first algorithm had been suggested in 1975 by Patil and a second one has been made available by Fahmy and Bellétoile in 2017.
Cochran's Q test is based on the following assumptions:
- If the large sample approximation is used (and not the exact distribution), b is required to be "large".
- The blocks were randomly selected from the population of all possible blocks.
- The outcomes of the treatments can be coded as binary responses (i.e., a "0" or "1") in a way that is common to all treatments within each block.
- The Friedman test or Durbin test can be used when the response is not binary but ordinal or continuous.
- When there are exactly two treatments the Cochran Q test is equivalent to McNemar's test, which is itself equivalent to a two-tailed sign test.
- William G. Cochran (December 1950). "The Comparison of Percentages in Matched Samples". Biometrika. 37 (3/4): 256–266. doi:10.1093/biomet/37.3-4.256. JSTOR 2332378.
- Conover, William Jay (1999). Practical Nonparametric Statistics (Third ed.). Wiley, New York, NY USA. pp. 388–395. ISBN 9780471160687.
- National Institute of Standards and Technology. Cochran Test
- Mohamed M. Shoukri (2004). Measures of interobserver agreement. Boca Raton: Chapman & Hall/CRC. ISBN 9780203502594. OCLC 61365784.
- Kashinath D. Patil (March 1975). "Cochran's Q test: Exact distribution". Journal of the American Statistical Association. 70 (349): 186–189. doi:10.1080/01621459.1975.10480285. JSTOR 2285400.
- Fahmy T.; Bellétoile A. (October 2017). "Algorithm 983: Fast Computation of the Non-Asymptotic Cochran's Q Statistic for Heterogeneity Detection". ACM Transactions on Mathematical Software (TOMS). 44 (2): 1–20. doi:10.1145/3095076.