Talk:Regression testing

From Wikipedia, the free encyclopedia
Jump to: navigation, search
WikiProject Computing (Rated Start-class)
WikiProject icon This article is within the scope of WikiProject Computing, a collaborative effort to improve the coverage of computers, computing, and information technology on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.
Start-Class article Start  This article has been rated as Start-Class on the project's quality scale.
 ???  This article has not yet received a rating on the project's importance scale.
 

Older discussions[edit]

Does anyone know how to document regression testing or any other kind of testion for that matter?Stanko 19:10, 27 September 2006 (UTC)Stanko

I removed this line, as it clearly belongs in talk and not in the article itself: Remaining questions - who invented the term? Who was the first to use it systematically?

And I'm not sure about this quote (I left it in, but added quotations marks to make clear that it's a quote.)

"As a consequence of the introduction of new bugs, program maintenance requires far more system testing per statement written than any other programming. Theoretically, after each fix one must run the entire batch of test cases previously run against the system, to ensure that it has not been damaged in an obscure way. In practice, such regression testing must indeed approximate this theoretical idea, and it is very costly." -- Fred Brooks, Mythical Man Month (p 122)

-starwed

References[edit]

The article would really need some references/external links. Anyone having some? Cheers - David Björklund 11:21, 13 Mar 2005 (UTC)

Relation with extreme programming[edit]

The last paragraph states that "regression testing is an integral part of the extreme programming software development methodology". This is IMO abusive since it is part of many, if not all, of those methodologies. Furthermore, the article about extreme programming does not mention regression testing, which tends to prove my point. Regression testing is merely a software engineering trick that many of us use (whereas all should). - 81.56.146.158 15:58, 25 October 2005 (UTC)

I concur. Extreme or Agile programming has a different view with regards to testing in that it preaches to write tests before implementing features, something which is less common and seldom explicitly preached in other development methods. In other methodologies it's not necessarily discouraged or not practiced; it's just not explict as one of the most import principles. Wouter Lievens 08:06, 7 April 2006 (UTC)
I think that eXtreme Programming preaches to write tests before implementing features for a few reasons. One of those reasons is so that you have a full suite of unit tests to run as a regression test at any time. And I think that the reason for that is because in extreme situations, there can be no tolerance for slippage, or regression. Every update to the code must be a step forward.
Like 81.56.146.158 mentioned, regression testing is a trick that many of us use and most of us should. But I think that eXtreme Programming demands it. Maybe it has more to do with Test Driven Development. And maybe someone should update the eXtreme Programming article to mention regression testing. DRogers 12:28, 19 March 2007 (UTC)

Why does this apply only to software?[edit]

I hoped to see a definition of regression testing that applied to systems in general, not just software. Is there some reason this definition shouldn't be applied to entire systems, subsystems, and integrated hardware/software units? —The preceding unsigned comment was added by 192.45.72.26 (talkcontribs) 23:53, 25 September 2006 (UTC).

Well 192.45.72.26, the best answer I can give is that software people started this article and software people maintain it. Regression testing as applied to software is all I know. If you think that it applies to something else, I encourage you to start an article. DRogers 12:19, 19 March 2007 (UTC)

Regression testing is one of the key elements when designing integrated circuits, ASICs and FPGA based designs. In this field there are quite a lot of specialiced tools on the marked to help designers with it. On this top 10 asic design and verfication list, regression testing is on the second place. the problem is a bit, that this article belongs to the software testing portal (which is ok) and that there is no article about asic design methods and verification where it would also belong to. --84.20.36.138 (talk) 07:13, 20 April 2011 (UTC)

It belongs to Wikipedia. If you want to add a section on other areas, feel free to. You could also create a different article. It just sounds like there are different methods involved though and the concept remains the same. --Walter Görlitz (talk) 13:52, 20 April 2011 (UTC)

Important questions[edit]

The article does not answer 2 important questions:

  1. What does the word "regression" in this context mean?
  2. And why are the tests called "regression tests"?

I recommend you to read the much better article "Regression Testing" at http://www.wrox.com/WileyCDA/Section/id-291252.html by Adam Kolawa

—The preceding unsigned comment was added by 195.72.132.1 (talkcontribs) 09:58, 28 March 2007 (UTC1)

regression as opposed to progression - a step back not forward.
From Latin 'gressus' - Perfect active participle of gradior (“‘step, go, walk’”).
--87.194.174.252 (talk) 11:00, 17 September 2009 (UTC)

(NJgeezer (talk) 15:51, 16 November 2010 (UTC)) The Wiley article has this definition: "Regression testing identifies when code modifications cause previously-working functionality to regress, or fail..." The important point here is that the existing def in this wiki article does not explain the added nuance of "regression" , just "testing". The function of regression testing is not to test the change itself (commonly known as unit testing), but to ensure through broader testing that the change has not caused a different bug. This will almost always involve programs or modules outside the scope of the actual code changes. e.g. a change to a DB extract may have resolved a missing country's records, but those "new" records may contain data that causes a completely separate, downstream system to crash since their code does not allow for that new country's currency code. So we are testing not that the change works, but that it has not caused any application regression (loss of functionality) i.e. a failure of previously working, unchanged code.

Types of regression[edit]

61.95.203.186, I removed your edits. First, I don't think that those are types of regression. Maybe they would belong under the risk mitigation section. And in that case, I think "Complete test suite repetition" and "Partial test repetition based on traceability and analysis of technical and business risks" cover your points. Does anyone disagree? DRogers 12:48, 23 May 2007 (UTC)

Added citations and a testing resource[edit]

Since this topic is lacking cited content, I added some from Albert Savoia and Kent Beck. Kent Beck is the father of XP and JUnit (although I deliberately avoided making the new content Java-specific and stuck with what would transcend multiple languages). Alberto Savoia has written extensively on unit testing, regression testing, and developer testing.

In order to avoid tying content to any language, I dropped "JUnit Factory" down to the bottom as an external link instead of making it a reference.

Cheers. MickeyWiki (talk) 16:25, 29 November 2007 (UTC)

first paragraph a bit mistakable?[edit]

I dont know what regression testing is - but as i understand the explanation here, it is a test applied just after changes to software have been made. These changes may be "normal" changes or bugfixes. If this definition is correct, i find the the second sentence "The intend of regression testing is to assure that a bug fix has been successfully corrected.." mistakable; this sounds to me like "it is a test, that is applied after trying to fix a bugfix". So there was a mistake _fixing a bug_? I think this is too special..

In the case that my interpretation is correct, i suggest following two sentences for the beginning:

Regression testing is any type of software testing that seeks to uncover software errors after changes to the program (e.g. bugfixes) have been made, by partially retesting the program. The intent of regression testing is to assure that a bugfix itself is correct, or more general, that a change did not introduce new bugs.

and maybe the next sentence should be shorted as to not become repetitive.


len48 —Preceding unsigned comment added by 95.119.140.35 (talk) 19:07, 30 December 2009 (UTC)

I second the motion but don't feel comfortable making the change myself. If you need more evidence: "Regression testing is performed after making a functional improvement or repair to the program." [The Art of Software Testing, by G. Myers]
75.87.186.225 (talk) 22:15, 22 October 2010 (UTC)
I updated it taking into account what has been noted here. AliveFreeHappy (talk) 23:10, 22 October 2010 (UTC)

Is this claim too bold?[edit]

In the corporate world, regression testing has traditionally been performed by a software quality assurance team after the development team has completed work. However, defects found at this stage are the most costly to fix.

This latter statement in the background section seems to be a bit odd and does not have any citation or reference. I would have thought defects found after QA would be even more expensive, but does agree that QA tests are more costly then dev team tests. — Preceding unsigned comment added by 95.45.199.134 (talk) 10:20, 10 June 2011 (UTC)

Both are correct. There have been several studies that show increasing cost for bug fixes at each stage of development. It is 100 time more expensive to fix the bug if found by the client than if found at the time of creating requirements. However, the phrase above just needs a little tweak: "defects found at this stage are the more costly to fix than if found earlier." --Walter Görlitz (talk) 13:50, 10 June 2011 (UTC)

Regression Testing vs Non-regression testing[edit]

Should this article be merged with Non-regression_testing? What is the difference between these two terms (I do not see any). Is one of the terms more correct than the other (I was always saying Regression Testing, but recently met people that always say Non-regression Testing)? — Preceding unsigned comment added by Loomchild (talkcontribs) 14:19, 10 January 2012 (UTC)

No. The concepts are different, and "NRT" is unsupported in the literature. --Walter Görlitz (talk) 15:32, 10 January 2012 (UTC)
Thanks for explanation and for raising this topic on NRT Talk page. --Loomchild (talk) 14:27, 11 January 2012 (UTC)

edits by anon Woodbridge, Connecticut[edit]

These edits are not correct. You want to find errors, not defects. The errors are created by developers and they are not defects until someone actually triggers them. Also, defect is a euphemism for a software bug which is described as "A software bug is the common term used to describe an error, flaw, mistake, failure, or fault in a computer program or system that produces an incorrect or unexpected result, or causes it to behave in unintended ways." This is why we should not be using defect. As for "or other software properties", what sort of weasel words are those? Functionality is all that is important in regression testing. There are no other relevant "software properties" (whatever that means). --Walter Görlitz (talk) 22:59, 28 February 2012 (UTC)

I have no hope of convincing Walter but here are some definitions that should illuminate the two issues. Emphasis added.

error – “a human ACTION that produces an incorrect result, such as software containing a fault" (ISO 24765:2010, Systems & Software Engineering Vocabulary)

defect – “a problem which, IF NOT CORRECTED, could CAUSE an application to either fail or to produce incorrect results” (Id.)

regression testing – “testing required to determine that a change to a system component has not adversely affected functionality, RELIABILITY OR PERFORMANCE, and has not introduced additional defects” (Id. Also: ISO 90003:2004, Software Engineering Guidelines for the Application of ISO 9001:2000 to Computer Software)

The other option is to argue with international standards. — Preceding unsigned comment added by 69.183.41.121 (talk) 02:44, 29 February 2012 (UTC)

It's not about convincing me, it's about not being vague and not using terms in incorrect ways. --Walter Görlitz (talk) 03:09, 29 February 2012 (UTC)

I gave my sources of authority. What are yours? — Preceding unsigned comment added by 69.183.41.121 (talk) 03:30, 29 February 2012 (UTC)

A library of a dozen books, fifteen years of Better Software Magazine and twenty years in the industry. Many of the testing articles on Wikipedia were either started or heavily edited early on by me.
The fact that you have sources don't mean that your additions are any more clear than what was there. Your addition of "or other software properties" is vague and your sources (which are not listed in the article per WP:V and WP:RS) don't help make it less vague. The fact that your ISO terminology differs from some of the rest of the industry doesn't mean that it's right, its one opinion. Check the software bug article to compare with consensus. --Walter Görlitz (talk) 03:35, 29 February 2012 (UTC)

ISO standards come into existence through world-wide consensus among experts in the subject domain. May I ask one more time for the precise sources for your def’ns of error, defect and regression testing, with quotation of these definitions, so everybody can compare authority and actual words. 69.183.41.121 (talk) 04:02, 29 February 2012 (UTC)

Check the software bug article. --Walter Görlitz (talk) 04:16, 29 February 2012 (UTC)

That's what I did and that's why I made the changes - just in the first sentence, for a start, the article needs it. If the present version is representative, nobody should be surprised by the sorry outcomes of much of the software engineering industry. Cheers, 69.183.41.121 (talk) 04:47, 29 February 2012 (UTC)

The only sorry thing is your addition of ambiguous statements. Please expand the "or other software properties" phrase or remove it. --Walter Görlitz (talk) 04:56, 29 February 2012 (UTC)

Expanded phrase, done. Thanks. 69.183.41.121 (talk) 05:05, 29 February 2012 (UTC)

Now that I have time to read the whole I understand. You're confusing functionality and functional testing, which is why you're comparing it to non-functional requirements. Functionality is not the same. Fixing. --Walter Görlitz (talk) 06:13, 29 February 2012 (UTC)

No, you don't understand the whole - but that's ok. Your edit sounds as if it comes from a German or Dutch speaker. No verbs at the end of a sentence. Try again. 69.183.41.121 (talk) 07:45, 29 February 2012 (UTC)

Fixed the incorrect change to the relative clause. --Walter Görlitz (talk) 14:15, 29 February 2012 (UTC)

Good to see “error” removed and “non-functional” introduced. The non-functional aspects are the “number one risk” in software engineering. (CrossTalk, April2007, p.12) Being such a risk the non-functionals need full attention by regression testing. 69.183.41.121 (talk) 15:55, 29 February 2012 (UTC)

But regression testing doesn't usually include non-functional testing despite it being the number one risk. We're not creating policy but should be reflecting actual use. --Walter Görlitz (talk) 15:57, 29 February 2012 (UTC)

You have to start moving to different circles. 69.183.41.121 (talk) 16:12, 29 February 2012 (UTC)