User talk:Oo d0l0b oo

Hello! I see that you disagree with the statements made previously and that you went ahead to revert several of the changes. Let's discuss.

The following papers and researchers agree that the coupling effect is one of two foundations making up mutation testing:

A. Jefferson Offutt. 1992. Investigations of the software testing coupling effect. ACM Trans. Softw. Eng. Methodol. 1, 1 (January 1992), 5-20
Yue Jia; Harman, M., "Constructing Subtle Faults Using Higher Order Mutation Testing," Source Code Analysis and Manipulation, 2008 Eighth IEEE International Working Conference on , vol., no., pp.249,258, 28-29 Sept. 2008
Maryam Umar, "An Evaluation of Mutation Operators For Equivalent Mutants," MS Thesis, 2006.
Smith B., "On Guiding Augmentation of an Automated Test Suite via Mutation Analysis," 2008.

You can see that Offutt's paper is one of them.

Also, the following presentations also agree that the coupling effect is critical to mutation testing:

Polo M. and Piattini M., "Mutation Testing: practical aspects and cost analysis," University of Castilla-La Mancha (Spain), Presentation, 2009.
Anderson S., "Mutation Testing", the University of Edinburgh, School of Informatics, Presentation, 2011.

I believe the confusion arises with respect to the *need* for tools that claim to do mutation testing to support the coupling effect.

You stated: "This definition of the coupling effect is simply wrong and likely to cause confusion."

and you changed it to:

"The coupling effect asserts that test data sets that detect simple types of faults are sensitive enough to detect more complex types of faults."

I'm afraid you are not only relying on old results (from 1992), but are actually reading a hypothesis that is left open, as asserted by Offutt himself in the conclusion:

"Whether killing n-order mutants means we can detect complex faults is yet to be determined."

Yue Jia; Harman, M. showed in their 2008 paper that indeed killing n-order mutants means we can detect complex faults. Hence, generating higher-order mutants is not only shown to be necessary for detecting complex faults, but also required by tools that claim to do any mutation testing. This is why the coupling effect is now known as a critical part (one of two pillars) of mutation analysis and, without it, the technique is incomplete.

Additionally, there is empirical evidence that tools that do not support higher-order mutants and therefore do not support the coupling effect (such as mutant and PITest) are unable to generate mutatants that would allow the user to detect rather simple blind spots in tests. In particular, the following gist shows that neither mutant nor PITest were able to generate "a mutant that would cause us to discover the need for this test", where the author is referring to a very simple blind spot in the tests. Such blind spot was only detected by a higer-order mutant. The gist is here: https://gist.github.com/omgm/69f91fe1eace6a6d681f

I hope this clarifies the confusion.

65.129.180.159 (talk) 05:33, 30 July 2014 (UTC)[reply]

I agree that the literature considers the coupling effect to be one of the foundations of mutation testing. The issue is that you are attempting to re-define it to mean something different.

Research over time may question whether the it is a valid hypothesis, but it does not redefine it. And the coupling effect is indeed a hypothesis, as Offut puts it in http://www.cs.gmu.edu/~offutt/rsrch/papers/coupl.pdf

"Since examples of complex faults that are not coupled to simple faults can be constructed, the coupling effect is probabilistic rather than absolute"

So it's a hypothesis that is never likely to be "disproved" as it makes only a weak assertion.

Harman's paper shows that some higher order mutants subsume first order mutants and Harman argues that it may be possible for tools to be more efficient if they generate HOM's. It does not logically follow that a tool must generate higher order mutants.

I suggest the article is amended to have a standard definition of the coupling effect as used in the peer reviewed literature - it is not clear where the current text originates from. It does not appear to come from any of the referenced sources.

You appear to be a proponent of HOM's, instead of describing tools as "partial" mutation testers, why not change this to be an objective statement as to whether or not they support higher order mutants? Oo d0l0b oo (talk) 09:01, 30 July 2014 (UTC)[reply]