Talk:Comparison of GUI testing tools

Page contents not supported in other languages.
From Wikipedia, the free encyclopedia

Junit[edit]

I don't think JUnit really belongs in this list. Regards, Ben Aveling 03:17, 10 August 2008 (UTC)[reply]

Eggplant[edit]

Eggplant needs to stay on this list. It's been around longer than Ranorex, and has more users. It's an important tool that needs to be in every tester's tool box. I've been using a combination of tools including Eggplant since about 2003. Ajfisher2 (talk) 22:30, 21 January 2010 (UTC)[reply]

Age is not a criteria. Number of users is. I'll remove Ranorex as well. Thanks for the head's up. The list isn't for advertising purposes, it's an example of important tools. --Walter Görlitz (talk) 23:04, 21 January 2010 (UTC)[reply]
BTW, the reason Ranorex was on the list was it had an article. Its gone now too. --Walter Görlitz (talk) 23:05, 21 January 2010 (UTC)[reply]

Putting Ranorex and Eggpalnt back. Creating a list based solely on number of users is a truly stupid criteria. It would be like me being paid solely for the number of bugs I find. There are many other things that should be considered when posting a tool besides number of users. Approach to the problem, technologies supported, licensing, cost, skill required of the tester...these are all things that are things I consider when adding a tool to my toolbox. By removing Eggplant, for example, you've removed half the tools on the list that support Mac OS. Are you a tester? I don't think you are, or else you would understand this.

So how many users does a tool need to have? My understanding of wikipedia is to provide information. I was trying to provide some. There is a link encouraging the expansion of this list. I'm going to remove that link because it's all about YOUR opinion of the list since it's impossible to add anything without you removing it and dismissing it out of hand. I have 3 or 4 tools that I would like to add in addition to Eggplant, but now I'm not because you've successfully made it a complete waste of time.

Eggplant certainly is a different tool, it works much, much differently than the rest of the tools on this list...which is why I thought it important to add. I hate to burst your bubble, but most of the tools here are listed for advertising purposes. It does things differently, so I thought it important. If you're so hell bent on removing tools because they don't have as many users as your tool of choice, you might as well just remove everything except QTP and RFT. Ajfisher2 (talk) 14:34, 22 January 2010 (UTC)[reply]

I am going to remove it one more time. This time, I would like you to create a page for the tool. If that page is standing in a week, feel free to link to that page from here. The fact that it has never appeared on the StickyMinds.com/Better Software Magazine tools survey as a top tool backs my point up. But then again, neither has Ranorex. The issue is that the page says notable, and we have to have some criteria for judging notability. The criteria I have chosen is popularity. If you would like to discuss a better criteria, feel free to do so, but do not place the tool back until there is some agreement on what it means to be a notable tool. I am offering two points: number of users and notability as determined by appearance in Wikiedia. --Walter Görlitz (talk) 14:45, 22 January 2010 (UTC)[reply]
It's also important to note that Wikipedia lists are not supposed to be a complete accounting of tools, etc. Besides there are already lists like that:
Eggplant is on three of the lists above. They state it's a Mac OS X specific QUI testing tool. --Walter Görlitz (talk) 18:52, 22 January 2010 (UTC)[reply]
Final thoughts for now: Please see WP:Links before adding what you think are good links. People have been discussing this for several years and have come up with some guidelines of what makes a good link and what doesn't. --Walter Görlitz (talk) 19:32, 22 January 2010 (UTC)[reply]

OK. I've put up a page for Eggplant. I also do not see anything on WP:Links that qualifies removal. But I will do the extra work. I think notable should include, in addition to the points mentioned above, maturity, and approaches that are outside what you might call "standard" for GUI tools. All the tools on that list use gray-box testing. Eggplant is a blackbox tool, and has a decent user base.Ajfisher2 (talk) 20:18, 25 January 2010 (UTC)[reply]

Excellent! Hope the page survives. It looks like it should. We need more people who love their tools like you do to create pages for them. --Walter Görlitz (talk) 21:01, 25 January 2010 (UTC)[reply]

What to add[edit]

I am reminded that this list should not contain external links by WP:NOTLINKFARM. Externally linked articles will be removed.

This should be a category list on articles no external links at all. It's very existence makes it a linkfarm
Agreed. There are a lot of lists of GUI testing tools, and many of the tools listed there are o longer maintained. --Walter Görlitz (talk) 20:43, 3 March 2010 (UTC)[reply]

A list for GUI testing tools is also available here http://www.testingfaqs.org/t-gui.html. Maybe somebody wants to add a tool at this article's list. I would like to, but I have no time! 194.219.52.200 (talk) —Preceding undated comment added 10:19, 14 July 2010 (UTC).[reply]

It's me again. I didn't see that you already had the above link at the "External Links" section. Never mind... :) 194.219.52.200 (talk) —Preceding undated comment added 10:22, 14 July 2010 (UTC).[reply]

QAliber[edit]

I was wondering if you can include QAliber to the list, it is an open source project at sourceforge (http://sourceforge.net/projects/qaliber/)

Please let me know what do you think, Regards, Benny —Preceding unsigned comment added by 194.90.236.138 (talk) 13:35, 4 March 2010 (UTC)[reply]

You would have to add an article for the tool first. You would then have to ensure that it stands any challenges regarding notability. Once the article is established, we'd be happy to have you add it here. --Walter Görlitz (talk) 14:50, 4 March 2010 (UTC)[reply]
I added an article for the tool, hope it will pass the notability challenge, let's see if it stays a few days first. Benny Cohen (talk) 10:21, 5 March 2010 (UTC)[reply]
I've added the QAliber tool to the list, since the QAliber article stayed for 4 days 20:17, 8 March 2010 (UTC)

Table?[edit]

This page would benefit from a table with attributes such as

  • platform tools runs on (windows, unix)
  • gui tech tool can test (Java, Swing, Awt, QT, GTK) — Preceding unsigned comment added by 199.46.196.231 (talk) 14:20, 6 July 2011 (UTC)[reply]
There aren't that many tools. It would be easy to click through to each. --Walter Görlitz (talk) 14:44, 6 July 2011 (UTC)[reply]
I made one large table instead and I made it sortable. It should be easier now. Walter Görlitz (talk) 03:25, 23 March 2014 (UTC)[reply]

List of GUI testing tools modification was rolled back[edit]

Hi Walter,

I added a and open source GUI testing tool called VANGA. I am a contributor to the project. Why did you roll back my adition?

Thanks,

Tervel — Preceding unsigned comment added by 217.108.170.8 (talk) 15:22, 1 December 2011 (UTC)[reply]

The Open Source product was removed because only products with articles are to be included in this list. --Walter Görlitz (talk) 15:39, 1 December 2011 (UTC)[reply]
If you want to create an article for the product please make sure it's notable first. --Walter Görlitz (talk) 16:00, 1 December 2011 (UTC)[reply]

what is the System Requirement of "Example" ?[edit]

In the table Open Source GUI testing tools some rows have an entry "Example".
Is this filler text that should be removed?

JamesThomasMoon1979 03:31, 18 February 2014 (UTC)[reply]

Removed. Walter Görlitz (talk) 05:19, 18 February 2014 (UTC)[reply]
Fixed one leftover.

split lower table, and maybe combine redundant names[edit]

Currently the article has FLOSS tools for GUI test, proprietary tools for GUI test, and a mixed table for test-automation. Can we split the third table, to give us four tables, and consistency?  :-)

  My other suggestion is a bit more invasive; there are many names listed under GUI test, and also under test automation; should we just make a checkmark-column, so that we can show which GUI test tools also support test automation? It seems less than useful to list tools twice. — 74.192.84.101 (talk) 05:35, 16 March 2014 (UTC)[reply]

possibilities of additional tools[edit]

Here are some tools that might be worth adding to the tables.

checklist used to search for possibilities

Mentioned on stackOverflow.

http://ifets.ieee.org/russian/depository/v13_i1/html/10.htm Master's thesis, 2010, Latvia. Appears to be published (with peer-review) as an IEEE conference-paper, but I don't grok the lingo, so I'm not positive here. Major and minor tools, FLOSS, used by a much wider variety of projects than the fancy payware (and often complementary but also competitive with the corp stuff).

Products from major corporations, which typically bought out some smaller company that created the product originally. IBM/HP/Borland + Oracle.

Niche products and new startups and similar.

http://bauhaus.cs.uni-magdeburg.de:8080/miscms.nsf/FEA8C8150500AA14C1257449004F79A9/D01B44988A710A28C1257AA000304EE7/$FILE/Bachelorarbeit%20Philipp%20Ernst.pdf hundred-plus-page "bachelor" thesis, 2012, Germany... note that this might actually be a higher-level degree, Germany has a bit of an odd naming-system for the educational setup. Listed a couple FLOSS tools, one from Mozilla.

Products from major corporations, which typically bought out some smaller company that created the product originally. IBM/HP/Borland + Siemens.

Niche products and new startups and similar.

http://urn.kb.se/resolve?urn=urn:nbn:se:mdh:diva-18701 , archive record at http://www.diva-portal.org/smash/record.jsf?pid=diva2:613817 hundred-page master's thesis, 2013, sweden, Major and minor tools, FLOSS, used by a much wider variety of projects than the fancy payware (and often complementary but also competitive with the corp stuff).

  • Selenium , FLOSS (Apache2)
  • Watir is an open–source testing tool based on Ruby libraries for automating the testing on web browsers. It is distributed under BSD license
  • FitNesse , Robert C. Martin (CPL)

Products from major corporations, which typically bought out some smaller company that created the product originally. IBM/HP/Borland + Microsoft + National Instruments.

Niche products and new startups and similar.

http://download.microsoft.com/download/E/E/8/EE802CF5-0E08-431F-BD45-2B938E70F625/Gartner_Magic_Quadrant_Testing_Jan2011.pdf

https://en.wikipedia.org/wiki/Category:Graphical_user_interface_testing

https://en.wikipedia.org/wiki/Category:Software_testing_tools

https://en.wikipedia.org/wiki/Category:Load_testing_tools

https://en.wikipedia.org/wiki/List_of_web_testing_tools

https://en.wikipedia.org/wiki/List_of_unit_testing_frameworks


  • HttpUnit (from List_of_unit_testing_frameworks#Internet), open source software testing framework, tests web sites (HTML/forms/JS/auth/redirects/cookies) without using a browser. (( Might be too "manual" for these tables? ))
  • HtmlUnit (from List_of_unit_testing_frameworks#Internet), open source headless web browser, allows high-level programmatic manipulation of websites/HTTPS/redirects/auth/links/forms/JS (from Java code), with test-automation for web apps the most common use. (( Might be too "manual" for these tables? ))




  • Certify Suite by Worksoft (from Gartner'11) automates quality assurance: functional validation, performance testing, load testing, etc.
  • QA Wizard Pro by Seapine_Software (from Gartner'11), automated functional-testing and load-testing[1]
  • CloudTest by SOASTA (from Gartner'11), cloud-based testing service for load testing (thousands of simulated users), performance testing, functional testing, and UI testing
  • TestDrive by Original Software (from Gartner'11), automatic software testing
  • GH Tester by IBM's Green_Hat_(software_company) (from Gartner'11), automated testing tool for distributed technologies such as SOA, BPM, and other middleware.
  • LISA Test by CA Technologies's Itko (from Gartner'11), high-volume load testing, automated functional testing and load testing for distributed apps (e.g. RIA)


Hope this helps. 74.192.84.101 (talk) 05:35, 16 March 2014 (UTC)[reply]

Notability again[edit]

I reverted your edit of the list of GUI testing tools. I see no reason that the list should not include a tool just because there is no corresponding article for it. It's a real tool with a real user community. I provided a link to the webpage and relevant information. Victorianist (talk) 02:10, 29 July 2014 (UTC)[reply]

The comment is clear:

<!-- IF YOU DON'T PAY ATTENTION TO THIS MESSAGE, YOUR EDIT WILL BE ROLLED BACK WITHOUT WARNING. Only place entries here that are links to actual Wikipedia articles about notable GUI testing tools. External links, redlinks, non-notable sites will be removed. If you have questions, use the talk page. Please try to keep entries in alphabetical order. Adding unnecessary links or text to any other section (such as the "References" section) will also be removed. Thanks. -->

The discussion here has been unanimous, however, if you want to approach a new consensus, this is where the discussion should happen not on my talk page. Don't try to change it by edit warring either. Walter Görlitz (talk) 04:10, 29 July 2014 (UTC)[reply]
Thank you for trying to make this article less useful by deleting references to GUI testing tools. It is a shame that small wikipedia contributions like adding a tool to this list is simply deleted rather than you leaving it so that someone else could improve it by taking the minimal time to create a stub article for the tool. Your scolding tone is also unnecessary. I've never been involved in a revert war so I'm just going to leave the article alone and it will continue to be what it is: incomplete, sadly out of date and therefore useless. In my opinion, this makes Wikipedia worse, not better. Victorianist (talk) 14:01, 30 July 2014 (UTC)[reply]
I also want to note that what you call "unanimous discussion" seems to be mostly YOUR comments and edits. I think someone should review your dictatorial control of the page. I'll be asking the administrators to take a look. You seem to be unusually aggressive with this page. Victorianist (talk) 14:03, 30 July 2014 (UTC)[reply]
Less useful, that's one opinion. WP:LINKFARM is another.
How would listing every product help improve Wikipedia?
How would listing every product help help Wikipedia contributors?
My actions have been supported by other editors. I simply happen to be the most active editor. I am no more aggressive with this page than I am on any page I have on my watch list.
And for the record, I'm not opposed to changing what lists like this contain, but trash talk and personal attacks doesn't help make the case for change. Walter Görlitz (talk) 14:12, 30 July 2014 (UTC)[reply]
Right, the policy you link to states the following: "External links or Internet directories. There is nothing wrong with adding one or more useful content-relevant links to an article; however, excessive lists can dwarf articles and detract from the purpose of Wikipedia. On articles about topics with many fansites, for example, including a link to one major fansite may be appropriate. See Wikipedia:External links for some guidelines." There is nothing excessive about linking to another tool that has the same stature and notability as some of the others here. I contend that the link I added to the table was "useful" and "content-relevant." There just aren't that many GUI testing tools in the world. There is no danger of it becoming a link farm. But you just go ahead and do what you like with the page. I'm done here. Victorianist (talk) 19:37, 30 July 2014 (UTC)[reply]
The excessive nature is when every back-yard hobbyist adds their tool regardless of notability to the list despite them thinking it's useful or content-relevant. There are many GUI Testing tools in the world, I have seen the lists elsewhere. Walter Görlitz (talk) 04:03, 31 July 2014 (UTC)[reply]
I have no relationship to Sikuli. Here's how it came that I added it to the list. I work for a company and we need an automated, cross-platform GUI testing tool that is also open source. There is one on your (note that I say "your" and not "Wikipedia's" list that I was interested in, but a colleague pointed me toward another tool that I had never heard of and that was Sikuli. I thought I would at least put a link to it on the page. But it seems that any arm-chair enthusiast without a higher degree in anything thinks that they know better about what should appear in a Wikipedia entry. You keep fighting the good fight. — Preceding unsigned comment added by Victorianist (talk • contribs) 03:54, 2 August 2014 (UTC)[reply]
Found out about Sikuli from a colleague after I looked into this Wikipedia list and didn't find any GUI testing tools based on optical recognition. Started rolling my own but luckily found out about Sikuli on time. Wikipedia needs a "USELESS" tag/banner on top of articles like this, so you know that the list is "incomplete" because people delete e.g. entries to MIT projects from it because random, not because nobody bothered to "help expanding it". — Preceding unsigned comment added by 189.26.125.29 (talk) 19:35, 6 May 2015 (UTC)[reply]
And also, I think you need to seriously edit this list. Most of the entries do not have citations, there's one redline link, and some of the others have some columns marked "unknown." I'll be cleaning up the list applying your own guidelines. Victorianist (talk) 03:58, 2 August 2014 (UTC)[reply]
I'm the one who added the citation needed tags. Feel free to remove the redlinks. Walter Görlitz (talk) 05:25, 2 August 2014 (UTC)[reply]
Walter has gained a reputation as a grade A douche nozzle for his maintenance of this page. — Preceding unsigned comment added by 24.24.202.165 (talk) 05:44, 19 November 2017 (UTC)[reply]
Funny. If I've done something wrong, take it to an administrator's forum. They'll listen. If you just want to troll, feel free to find another forum. Walter Görlitz (talk) 21:25, 20 November 2017 (UTC)[reply]

Article nominated for deletion[edit]

This article has been nominated for deletion before, in 2007 and the consensus at that time was that it should be deleted. The "criteria" of notability proposed in the talk page and in hidden comments in the article itself is that that each link should point to an already existing article on Wikipedia, but this does not establish notability. The list contains red links and none of the listed items points to verifiable sources or to sources that establish notability. — Preceding unsigned comment added by Victorianist (talk • contribs) 04:30, 2 August 2014 (UTC)[reply]

And the list was recreated in 2008. It currently only lists notable subjects despite your effort to turn it into a link farm and include other items. I really don't understand your logic.
Notability applies to subjects, I'm not sure it applies to lists. Walter Görlitz (talk) 05:33, 2 August 2014 (UTC)[reply]
Also, as I stated at the AfD page, the citation needed tags on each row are not related to whether the individual entry is or is not notable or reliably sourced simply that the attributes of each entry (testing system requirement, system under test requirement, GUI test, automation, and current version) is reliable information. It does not reflect whether the idea of such a list is or is not worthy of an article. Walter Görlitz (talk) 12:04, 2 August 2014 (UTC)[reply]
I think that you're quibbling. The articles for each tools listed here will also be proposed for deletion. Hardly any of them meet the criteria of notability or verifiability. By your criteria, a GUI tool must merely have an article, whether or not the article is a quality article. If that logic were allowed to continue, Wikipedia would contain both garbage articles and garbage indexes. I was not trying to create a link farm. I was trying to add another link. I now see that you're right. Wikipedia should be held to a higher standard. A list such as this is useless because it's not complete and the tools that it does list are here just because they have an article about them, whatever their level of notability. Victorianist (talk) 16:21, 2 August 2014 (UTC)[reply]
That's an interesting take. Let's let the AfD decide. Walter Görlitz (talk) 16:46, 2 August 2014 (UTC)[reply]

Sikuli / SikuliX[edit]

I'm trying to get Sikuli in the list. I tried to add an article so it wouldn't be rejected here, but I noticed lots of people are having trouble getting an article created due to Wiki's "notability" policy. Sikuli's a pretty significant tool to be missing from the list, notwithstanding all this pedantic procedural stuff. I've consulted at a number of Fortune 100 companies, and lots of them are using it. --Jrounceville (talk) 22:28, 3 August 2015 (UTC)[reply]

It's been added as an article, three times, but deleted due to lack of notability. The most recent discussion, Wikipedia:Articles for deletion/Sikuli, was July 2014. Based on what I'm seeing online, it still fails WP:GNG or any of the notability guidelines. WHile Fortune 100 companies may be using it, it's not use that determines notability, but people writing about it. It's not that they're using it, but how as well. Walter Görlitz (talk) 01:12, 4 August 2015 (UTC)[reply]
I just did a cursory examination and I'd also have a hard time saying that I can find enough for it to be notable. If you have some sources then please share them. AliveFreeHappy (talk) 19:04, 4 August 2015 (UTC)[reply]

Contested deletion[edit]

This article should not be speedy deleted as having no substantive content, because... it is useful — Preceding unsigned comment added by 66.194.230.26 (talk) 20:38, 16 May 2016 (UTC)[reply]

Agreed, this article is useful. Thanks! — Preceding unsigned comment added by 192.35.35.40 (talk) 13:50, 29 January 2020 (UTC)[reply]

GUI Testing Tools or General Testing Tools?[edit]

I came here because I have been doing research for my company on tools like these, but there are several tools on here now that are either only loosely related to GUIs, or not related at all. For example Ascentialtest is one. It is a regression testing tool as far as I can see, which has nothing to do with GUIs. There is also several web testing tools, where while web pages are GUIs, perhaps they should have their own category and list with a link to it at the bottom of this. e.g. "If you're looking for web testing tools, there is a list here:" After all, if you are looking for general GUI testing tools, a Web testing tool just doesn't cut it.

Just some feedback from me. I just had to make an account now, so I'll leave it to someone else in case people think I just came in here to troll. ChthonicOne (talk) 19:15, 24 February 2017 (UTC)[reply]

BrowserStack[edit]

Would BrowserStack be valid? Richardc020 (talk) 16:20, 17 May 2021 (UTC)[reply]

As a GUI testing tool? It tests websites, but not desktop GUIs, so no. Walter Görlitz (talk) 03:38, 18 May 2021 (UTC)[reply]
Although you could argue it being listed here, I'd have to agree with Walter. As far as I know, that would be more valid if the topic was for "websites" in general rather than "Web UI". Joedf (talk) 13:14, 18 May 2021 (UTC)[reply]

Tools to be added[edit]

Playwright and Cypress. 41.116.30.233 (talk) 19:59, 15 October 2022 (UTC)[reply]

Retrieved from "https://en.wikipedia.org/w/index.php?title=Talk:Comparison_of_GUI_testing_tools&oldid=1201079698"