Development testing: Difference between revisions
Providing additional details from the referenced work to demonstrate that no vendor referenced in the "Development Testing" section of this document is being excluded |
There are 19 on the list. The selection of these few is to promote them, especially Parasoft) (undo) |
||
| Line 41: | Line 41: | ||
Although many warn that static analysis alone should not be considered a silver bullet or panacea, most industry experts agree that static analysis is a proven method for eliminating many security, reliability, and performance defects. In other words, while static analysis is not the same as Development Testing, it is commonly considered a component of Development Testing.<ref name="blog.whitehatsec.com"/><ref>[http://www.crosstalkonline.org/storage/issue-archives/2009/200903/200903-Black.pdf Static Analyzers in Software Engineering] by Dr. Paul E. Black , CrossTalk: The Journal of Defense Software Engineering, March/April 2009</ref><ref>[http://eecatalog.com/dsp/2012/09/25/top-3-mistakes-with-static-analysis-for-embedded-and-safety-critical-development/ Top 3 Mistakes with Static Analysis for Embedded and Safety-Critical Development] by Arthur Hicken, EE Catalog, September 25, 2012</ref> |
Although many warn that static analysis alone should not be considered a silver bullet or panacea, most industry experts agree that static analysis is a proven method for eliminating many security, reliability, and performance defects. In other words, while static analysis is not the same as Development Testing, it is commonly considered a component of Development Testing.<ref name="blog.whitehatsec.com"/><ref>[http://www.crosstalkonline.org/storage/issue-archives/2009/200903/200903-Black.pdf Static Analyzers in Software Engineering] by Dr. Paul E. Black , CrossTalk: The Journal of Defense Software Engineering, March/April 2009</ref><ref>[http://eecatalog.com/dsp/2012/09/25/top-3-mistakes-with-static-analysis-for-embedded-and-safety-critical-development/ Top 3 Mistakes with Static Analysis for Embedded and Safety-Critical Development] by Arthur Hicken, EE Catalog, September 25, 2012</ref> |
||
==Development Testing Market== |
|||
Industry analysts reported the following regarding the Development Testing market: |
|||
"Development testing is placing a focus on testing and quality from the beginning of the |
|||
lifecycle and augmenting traditional QA testing. |
|||
* [[Coverity]] brings clarity and focus to the development testing category. By delivering solutions for the development phase of the lifecycle, Coverity is championing the notion of quality earlier in the lifecycle and throughout complex supply chains. |
|||
* [[Parasoft]] offers solutions in development testing as part of an integrated end-to-end testing offering and its historic focus on Automated Defect Prevention to identify and remove defects as early in the lifecycle as possible. |
|||
* [[Microsoft]] provides a developer unit testing framework with advanced capabilities. |
|||
* [[SmartBear]] provides peer code review with an audit trail for compliance."<ref>{{cite web|last= Lanowitz |first= Theresa| title= voke Market Mover Array Report: Testing Platforms |date=2012-06-05| accessdate=2013-04-05|url= http://www.vokeinc.com/news-and-events/press-releases/view/571-new-voke-research-finds-mobility-cloud-embedded-software-and-lifecycle-virtualization-driving-dramatic-shifts-in-testing-market.html }}</ref> |
|||
==See also== |
==See also== |
||
Revision as of 06:28, 6 April 2013
Development Testing is a software development process that involves synchronized application of a broad spectrum of defect prevention and detection strategies in order to reduce software development risks, time, and costs.
Depending on the organization's expectations for software development, Development Testing might include static code analysis, data flow analysis metrics analysis, peer code reviews, unit testing, code coverage analysis, traceability, and other software verification practices.
Development Testing Overview
Development Testing is performed by the software developer or engineer during the construction phase of the software development lifecycle.[1]
Rather than replace traditional QA focuses, it augments it.[2] Development Testing aims to eliminate construction errors before code is promoted to QA; this strategy is intended to increase the quality of the resulting software as well as the efficiency of the overall development and QA process.[3]
Development Testing Applications and Benefits
Development Testing is applied for the following main purposes:
- Quality assurance—To improve the overall development and test process by building quality and security into the software (rather than trying to test defects/vulnerabilities out).
- Industry or Regulatory Compliance—To achieve compliance with industry or regulatory compliance initiatives (e.g.., FDA, IEC 62304, DO-178B, DO-178C, ISO 26262, IEC 61508, etc.) that commonly require strict risk reduction as well as bidirectional requirements traceability (e.g., between requirements, tests, code reviews, source code, defects, tasks, etc.)[3]
VDC research reports that the standardized implementation of Development Testing processes within an overarching standardized process not only improves software quality (by aligning development activities with proven best practices) but also increases project predictability.[4] voke research reports that Development Testing makes software more predictable, traceable, visible, and transparent throughout the software development lifecycle.[2]
Key Principles of Development Testing
In each of the above applications, Development Testing starts by defining policies that express the organization's expectations for reliability, security, performance, and regulatory compliance. Then, after the team is trained on these policies, Development Testing practices are implemented to align software development activities with these policies.[5] These Development Testing practices include:
- Practices that prevent as many defects as possible through a Deming-inspired approach that promotes reducing the opportunity for error via root cause analysis.
- Practices that expose defects immediately after they are introduced—when finding and fixing defects is fastest, easiest, and cheapest.[3][6]
The emphasis on applying a broad spectrum of defect prevention and defect detection practices is based on the premise that different Development Testing techniques are tuned to expose different types of defects at different points in the software development lifecycle, so applying multiple techniques in concert decreases the risk of defects slipping through the cracks.[3] The importance of applying broad set of practices is confirmed by Boehm and Basili in the often-referenced "Software Defect Reduction Top 10 List."[5]
Development Testing and Static Analysis
The term "Development Testing" has occasionally been used to describe the application of static analysis tools. Numerous industry leaders have taken issue with this conflation because static analysis is not technically testing; even static analysis that "covers" every line of code is incapable of validating that the code does what it is supposed to do—or of exposing certain types of defects or security vulnerabilities that manifest themselves only as software is dynamically executed.[7][8][9]
Although many warn that static analysis alone should not be considered a silver bullet or panacea, most industry experts agree that static analysis is a proven method for eliminating many security, reliability, and performance defects. In other words, while static analysis is not the same as Development Testing, it is commonly considered a component of Development Testing.[9][10][11]
See also
References
- ^ McConnell, Steve (2004). Code Complete (2nd ed.). Microsoft Press. ISBN 0-7356-1967-0.
- ^ a b voke Market Mover Array Report: Testing Platforms by Theresa Lanowitz, Lisa Dronzek, voke, June 05, 2012
- ^ a b c d Kolawa, Adam (2007). Automated Defect Prevention: Best Practices in Software Management. Wiley-IEEE Computer Society Press. ISBN 0-470-04212-5.
{{cite book}}: Unknown parameter|coauthors=ignored (|author=suggested) (help) - ^ "Automated Defect Prevention for Embedded Software Quality" white paper by VDC Research
- ^ a b Great expectations for development—with policy automation by Wayne Ariola, SD Times, July 28, 2011 Cite error: The named reference "sdtimespolicy" was defined multiple times with different content (see the help page).
- ^ Rethinking Software Development, Testing and Inspection by Matthew Heusser , CIO, February 1, 2012
- ^ Static Analysis isn’t Development Testing by Jim Bird, Building Real Software, January 6, 2012
- ^ Static or Dynamic Application Security Testing? Both! by Neil MacDonald, Gartner Blog Network, January 19, 2011
- ^ a b Mythbusting: Static Analysis Software Testing – 100% Code Coverage by Jeremiah Grossman , White Hat Security Blog, March 22, 2011
- ^ Static Analyzers in Software Engineering by Dr. Paul E. Black , CrossTalk: The Journal of Defense Software Engineering, March/April 2009
- ^ Top 3 Mistakes with Static Analysis for Embedded and Safety-Critical Development by Arthur Hicken, EE Catalog, September 25, 2012