Cantata++

From Wikipedia, the free encyclopedia
Jump to: navigation, search

Cantata++, or simply Cantata in newer versions, is a proprietary, commercial computer program for software testing, specifically unit testing and integration testing,[1] and code coverage at run time of C and C++ programs.[2][3] It is sold by QA Systems, and was formerly a product of IPL Information Processing Ltd.[4]

Overview[edit]

Cantata belongs to a category of code-driven unit testing frameworks used for dynamic testing of software. These tools bring compiled test cases to execution with the linked-in code under test. The test executable will run on a computing platform, a native operating system, or a target platform. Integration of debuggers is optional. Some but not all of these frameworks are based on xUnit.[5] Cantata is a proprietary framework not based on xUnit, and comprises: executables for Linux and Windows native operating systems, C and C++ libraries compiled for the test execution platform, and an Integrated development environment (IDE). However, existing xUnit tests can be embedded into Cantata++ as one or more Cantata++ testcases. Where code is unit tested with open source xUnit tools providing an acceptable degree of confidence, one option for companies is to replace the existing tests with an automatically generated set of Cantata++ baseline unit tests.

The IDE of Cantata is based on Eclipse. Furthermore, there is a Cantata Eclipse plug-in available for IDEs based on Eclipse, e.g. those of compiler vendors. Conversely, plug-ins available for Eclipse can be installed into the Cantata IDE. Because there is a command line interface present for Cantata, it is suitable for use in test automation and continuous integration frameworks. Cantata provides automation for test vector generation, a test harness, test drivers (test scripts in the C and C++ programming language), control of function call interfaces (via test stub simulations and wrapper function interceptions), building test executables and running them to produce test reports of actual against expected software behavior and source code coverage in XML, HTML, ASCII text formats, and diagnostic displays in the Cantata IDE. Cantata also performs static analysis which generates approximately 300 software metrics calculated on C and C++ source code in Certified Server Validation (CSV) format. Typical users are either developers of application software programmed in C and C++, for general-purpose computers, or of system software designed for embedded systems and mobile devices, or independent validation and verification (IV&V) engineers testing others’ code. Note that use of Cantata requires understanding of C/C++ code. Cantata is developed by QA Systems GmbH which acquired the product and retained its development staff from IPL Information Processing Ltd in 2012, at the same time founding QA Systems Ltd in Bath (UK).[6]

Industrial Use[edit]

The development and verification of functional safety software in many industries is governed by international regulatory standards. Therein requirements can be found which relate to functional Safety and how to ensure a certain Safety Integrity Level (SIL). Each of the standards below highly recommends or mandates the use of unit testing and integration testing with a code-driven testing framework:

  • Aerospace (Radio Technical Commission for Aeronautics: DO-178B and DO-178C : Software Considerations in Airborne Systems and Equipment Certification)
  • Automotive (ISO 26262: Road vehicles – Functional safety)
  • Energy/Nuclear Power (IEC 60880 : Nuclear power plants – instrumentation and control systems important to safety – Software aspects for computer based systems performing category A functions)
  • Industrial Automation (IEC 61508 : Functional Safety of Electrical/Electronic/Programmable Electronic Safety-related Systems)
  • Medical Devices (IEC 62304 : Medical device software – Software lifecycle processes)
  • Railways (EN 50128 : Railway applications – Communications, signalling and processing systems – Software for railway control and protection systems)

Cantata may be used in each of these sectors to meet the verification requirements of the regulatory standard. The Cantata tool has been independently classified and certified by the functional safety certification body SGS-TÜV GmbH, as “usable in the development of safety related software”[7] to the highest safety integrity level in each of the above standards. The mapping of verification and validation requirements in each of the above standards for functional software testing, robustness testing and structural testing (code coverage) to the capabilities of Cantata are available from the vendor, together with tool certification kits for IS0 26262, IEC 60880, IEC 61508, IEC 62304 and EN 50128, and tool qualification kits for DO-178B and DO-178C/DO-330.[2]

Mode of operation[edit]

The Cantata software library is configured (using a component of the tool) to be compiled and linked with the code under test as a test executable for running on the desired platform. Each library is therefore platform specific, and the user must select the correct compiler configuration within the tool. In the Eclipse IDE the Cantata user selects source code files to be tested, which when compiled are parsed with a built-in a parser from Edison Design Group.

A copy of the SUT(System under test) is called and controlled from a Cantata test script in portable C/C++ code, and contains a program entry point and test cases to call individual functions or methods. A copy of the SUT is used because Cantata uses source code instrumentation for white-box testing[8] otherwise prevented by C/C++ encapsulation (object-oriented programming) and for code coverage. Inputs to these functions (their parameters and other data accessible to them) are set in each test case. The user may select test case vector generation using: a default value test case for each function or method in the SUT, path solving test cases attempting to exercise all branches through the SUT, test cases with predefined sets of parameter and data values, or set inputs directly themselves (singularly or in tables of input values).

Cantata generates controls for function calls made by the SUT, so the user may select to control these function calls by simulating them using test stubs (otherwise known as Mock Objects), or by intercepting them with wrapper functions. The actions of stubs or wrappers on each call may be varied to perform setting and/or checking of parameters and data, and the sequence of such controlled function calls may be set by the user for each test case. The user selects test script checks on expected behavior and outputs of the SUT. In Cantata these test script checks are implemented as calls to the Cantata library, and may also compare to targets for code coverage.

When a Cantata test executable is built and executed, the SUT is called by the Cantata test script. Behaviour and outputs of the SUT are compared to those which are expected, and reported to the user in HTML and ASCII formats, together with Eclipse Graphical user interface views for detailed diagnostics of test results and code coverage. The Cantata code coverage analysis [9] may also be used to measure execution of C/C++ source code when an application is run (as opposed to being exercised by a Cantata test script). Cantata code coverage metrics measure basic coverage metrics such as Function entry-point coverage, Statement coverage, Branch (decision) coverage, as well as both the Unique Cause and Masking variants of Modified Condition/Decision Coverage, and others such as Relational Operator coverage and Loop coverage. Cantata static analysis (generation of code metrics) is used separately to dynamic testing of the SUT, as its primary purpose is to assess the quality (maintainability and testability) of source code , and estimate effort required to dynamically test it.

Baseline Testing[edit]

Cantata++ Baseline Testing creates complete Cantata++ unit test scripts from source code using an algorithm that automatically generates Cantata++ test case vectors that exercise all code paths, check data, parameters and call orders.[10] The definition of „all code paths‟ used to generate the baseline test cases is determined by selecting the metric types (Entry-point, Statement, Decision or unique cause MC/DC) in a code coverage Rule Set.

Literature[edit]

  • Peter Liggesmeyer: Software-Qualität: Testen, Analysieren und Verifizieren von Software. Spektrum, Akademischer Verlag, Heidelberg, Berlin, 2002, ISBN 3-8274-1118-1
  • Rune Winther, Bjoern Axel Gran, Gustav Dahll: "Computer Safety, Reliability, and Security: 24th International Conference, SAFECOMP 2005, Fredrikstad, Norway, September 28–30, 2005, Proceedings." Springer-Verlag GmbH, 2005, ISBN 3-5402-9200-4

External links[edit]

References[edit]

  1. ^ Master Thesis "A survey classification of software testing tools" Website of LAPPEENRANTA University of Technology, called 6-23-14
  2. ^ a b Dale, Chris; Anderson, Tom (2009-01-13). Safety-Critical Systems: Problems, Process and Practice: Proceedings of the Seventeenth Safety-Critical Systems Symposium Brighton, UK, 3 - 5 February 2009. Springer. pp. 216–. ISBN 9781848823495. Retrieved 3 June 2014. 
  3. ^ Wieczorek, Martin; Meyerhoff, Dirk; Baltus, B. (2001-02-09). Software Quality: State of the Art in Management, Testing, and Tools. Springer. pp. 267–. ISBN 9783540414414. Retrieved 3 June 2014. 
  4. ^ P, Mathur Aditya (2008). Foundations of Software Testing. Pearson Education India. pp. 489–. ISBN 9788131707951. Retrieved 3 June 2014. 
  5. ^ Cantata Feature Brief for Embedded xUnit tests Website of QA-Systems, called 6-30-14
  6. ^ Online Magazine "TEST– Innovation for software quality" Vol. 4, Issue 2: April 2012 ,called 11-7-14
  7. ^ Report about Cantata & Ada TEST 95 Website of Meteonic, called 7-2-14
  8. ^ White box testing concepts by Satish Mishra Website of "Humboldt University Berlin" called 7-1-14
  9. ^ Cantata Technical Brief Website of "Quality Checked Software Ltd." called 7-2-14
  10. ^ Cantata Baseline Testing Cantata Baseline Testing Feature Website of QA-Systems, called 7-11-14