Metamorphic testing

From Wikipedia, the free encyclopedia
Jump to: navigation, search

Metamorphic Testing (MT)[1] [2] is a software testing technique that attempts to alleviate the test oracle problem. A test oracle is the mechanism by which a tester can determine whether a program has failed. A test oracle problem occurs when it is difficult to generate expected outputs from selected test cases, or when it is difficult to determine whether the actual outputs agree with the expected outputs.

Metamorphic testing takes an original test case with unknown expected output and change it into a “follow-up” test case. For example, the expected output of a function that computes sin x to 100 significant figures is unknown. A metamorphic relation (MR) is a necessary property of the function, such as “sin x = sin (π − x)”. Hence, the program under test should have the same expected output for an original test case “0.1234” and a follow-up test case “π − 0.1234”. If the actual outputs from the program under test for these two test cases are different, it indicates a failure of the implementation.

An MR does not need to be an equation but can be a general relation. Thus, metamorphic testing can be applied to any problem in which a necessary property can be formulated. Some examples of successful applications include partial differential equations,[3] ubiquitous computing,[4] services computing,[5] imaging software,[6] bioinformatics,[7] software product lines,[8] and web search engines.[9]

References[edit]

  1. ^ First introduced in T.Y. Chen, S.C. Cheung, and S.M. Yiu, "Metamorphic testing: a new approach for generating next test cases", Technical Report HKUST-CS98-01, Department of Computer Science, Hong Kong University of Science and Technology, Hong Kong (1998).
  2. ^ First journal paper: T.Y. Chen, T.H. Tse, and Z.Q. Zhou, "Fault-based testing without the need of oracles", Information and Software Technology, vol. 45, no. 1, pp. 1–9, (2003).
  3. ^ T.Y. Chen, J. Feng, and T.H. Tse, "Metamorphic testing of programs on partial differential equations: a case study", in Proceedings of the 26th Annual International Computer Software and Applications Conference (COMPSAC '02), IEEE Computer Society, Los Alamitos, CA, pp. 327−333 (2002).
  4. ^ T.H. Tse, S.S. Yau, W.K. Chan, H. Lu, and T.Y. Chen, "Testing context-sensitive middleware-based software applications", in Proceedings of the 28th Annual International Computer Software and Applications Conference (COMPSAC '04), vol. 1, IEEE Computer Society, Los Alamitos, CA, pp. 458−465 (2004).
  5. ^ W.K. Chan, S.C. Cheung, and K.R.P.H. Leung, "A metamorphic testing approach for online testing of service-oriented software applications", International Journal of Web Services Research 4 (2): 60−80 (2007).
  6. ^ R. Guderlei and J. Mayer, "Towards automatic testing of imaging software by means of random and metamorphic testing", International Journal of Software Engineering and Knowledge Engineering 17 (6): 757−781 (2007).
  7. ^ T.Y. Chen, J.W.K. Ho, H. Liu, and X. Xie, "An innovative approach for testing bioinformatics programs using metamorphic testing", BMC Bioinformatics 10: article no. 24 (2009).
  8. ^ S. Segura, R.M. Hierons, D. Benavides, and A. Ruiz-Cortes, "Automated test data generation on the analyses of feature models: a metamorphic testing approach", in Proceedings of the 3rd International Conference on Software Testing, Verification, and Validation (ICST '10), IEEE Computer Society, Los Alamitos, CA, pp. 35−44 (2010).
  9. ^ Z.Q. Zhou, S. Zhang, M. Hagenbuchner, T.H. Tse, F.-C. Kuo, and T.Y. Chen, "Automated functional testing of online search services", Software Testing, Verification and Reliability 22 (4): 221−243 (2012).