Pseudoreplication

"Pseudoreplication is defined as the use of inferential statistics to test for treatment effects with data from experiments where either treatments are not replicated (though samples may be) or replicates are not statistically independent." The error described by this term arises when treatments are assigned to units that are subsampled and the treatment F-ratio in an analysis of variance (ANOVA) table is formed with respect to the residual mean square rather than with respect to the among unit mean square. The F-ratio relative to the within unit mean square is vulnerable to the confounding of treatment and unit effects, especially when unit number is small (e.g. four tank units, two tanks treated, two not treated, several subsamples per tank). The error is avoided by forming the F-ratio relative to the among unit mean square in the ANOVA table (tank MS in the example above). Pseudoreplication, as originally defined, is a special case of inadequate specification of random factors where both random and fixed factors are present.

Replication

Replication increases the precision of an estimate, while randomization addresses the broader applicability of a sample to a population. Replication must be appropriate: replication at the experimental unit level must be considered, in addition to replication within units.

Hypothesis testing

Statistical tests (e.g. t-test and the related ANOVA family of tests) rely on appropriate replication to estimate statistical confidence. Tests based on the t and F distributions assume homogeneous, normal, and independent errors. Correlated errors can lead to false precision and p-values that are too small.

Types

Hurlbert (1984) defined four types of pseudoreplication.

• Simple pseudoreplication (Figure 5a in Hurlbert 1984) occurs when there is one experimental unit per treatment. Inferential statistics cannot separate variability due to treatment from variability due to experimental units when there is only one measurement per unit.

• Temporal pseudoreplication (Figure 5c in Hurlbert 1984) occurs when experimental units differ enough in time that temporal effects among units are likely, and treatment effects are correlated with temporal effects. Inferential statistics cannot separate variability due to treatment from variability due to experimental units when there is only one measurement per unit.

• Sacrificial pseudoreplication (Figure 5b in Hurlbert 1984) occurs when means within a treatment are used in an analysis, and these means are tested over the within unit variance. In Figure 5b the erroneous F-ratio will have 1 df in the numerator (treatment) mean square and 4 df in the denominator mean square(2-1 = 1 df for each experimental unit). The correct F-ratio will have 1 df in the numerator (treatment) and 2 df in the denominator (2-1 = 1 df for each treatment). The correct F-ratio controls for effects of experimental units but with 2 df in the denominator it will have little power to detect treatment differences.

• Implicit pseudoreplication occurs when standard errors (or confidence limits) are estimated within experimental units. As with other sources of pseudoreplication, treatment effects cannot be statistically separated from effects due to variation among experimental units.

Notes

Hurlbert; reported 'pseudoreplication' in 48% of the studies he examined, that used inferential statistics. When time and resources limit the number of experimental units, and unit effects cannot be eliminated statistically by testing over the unit variance, it is important to use other sources of information to evaluate the degree to which an F-ratio is inflated by unit effects.