Game testing
From Wikipedia, the free encyclopedia
 |
| Part of a series on: |
| Video game industry |
|---|
|
Activities/jobs
|
|
Topics
|
Game testing refers to the process of software testing for video games. The formal process, performed by professional game testers, begins well into game development, depending on the game's allotted budget for testing. High-budget flagship titles (called "AAA", pronounced "triple-A") may begin testing with the first playable alpha, while low-budget and casual games might not enter testing until a release candidate (RC) is ready. Professional game testing may involve general playtesting but should not be confused with "public betas" or "stress tests" in which players simply do as they please and report bugs that are found incidentally. Public releases such as these are almost certainly late betas or RCs.
Contents |
[edit] Areas of testing
[edit] Functionality
This is the type of testing that is most commonly associated with the phrase "game testing", as it usually entails playing the game in some form. Functionality testing does not require much in the way of technical knowledge, though understanding basic concepts about programming can give testers insight about the possible cause of a problem, which both reduces the amount of work they have to do, and generally makes their reports easier to understand by the developer. Functionality testers look for general problems within the game itself or its user interface. Major issues include stability (does the game crash?), correctness of game mechanics (does the sword deal damage?), and integrity of game assets (are textures corrupted?).
[edit] Compliance
In large part, compliance testing is the raison d'etre for the existence of game testing labs. First-party licensors for console platforms (e.g. Nintendo, Sony, Microsoft) have strict technical requirements for each title that is licensed for their platforms. Sony publishes a Technical Requirements Checklist (TRC), while Microsoft publishes Technical Certification Requirements (TCR), and Nintendo publishes a set of "guidelines"(Lot Check) that serve a similar purpose. Some of these requirements are highly technical in nature and fall outside the scope of the game tester; however, other parts, most notably the formatting of standard error messages, handling of memory card data, and handling of legally trademarked and copyrighted material, do fall within the responsibility of the game testers to test. If even a single violation is found by the first party during submission for license approval, the game can be rejected (kicked back) to the publisher, possibly costing tens of thousands of dollars in additional testing and resubmission fees. In addition, the delay may cause the title to miss an important launch window, potentially costing the publisher even larger sums of money. The title cannot be published for the system without first-party approval, so it is in the publisher's financial interest to ensure that the title is tested for compliance as rigorously and as frequently as possible.
The requirements themselves are proprietary documents released to developers and publishers under confidentiality agreements. They are not available for the general public to review, although familiarity with these standards is considered a valuable skill to have as a tester.
Compliance may also refer to regulatory bodies such as the ESRB, if the game targets a particular content rating. Testers must flag "objectionable content" in the game that may be inappropriate for the desired rating. As with licensing, games that do not receive the desired rating must be re-edited, retested, and resubmitted at additional cost.
[edit] Compatibility
Compatibility testing is normally required only for PC titles, and testers assigned to this task are expected to have at least basic knowledge of PC hardware. An individual who can assemble their own PCs and setup an operating system is minimally qualified for this job. Compatibility testing differs from functionality testing in that the tester does not really care whether the game itself is correct, but simply that it runs on different configurations of hardware. The tester is usually provided with multiple PCs and/or extra parts. This collection of hardware encompasses various brands of CPUs, video cards, sound cards, and input peripherals such as gamepads and joysticks, and possibly other components as well. The tester is also expected to cross-test different hardware configurations with different operating systems. In addition to a basic pass/fail test, the tester may also be asked to evaluate performance. The results of these performance tests will be used to determine the game's advertised minimum system requirements. In some cases, compatibility or performance issues may be fixed by the developer, especially if a popular brand and model of a certain component is affected. In other cases, especially with legacy hardware or software, support will simply be dropped, and the game's system requirements will reflect this.
[edit] Localization
While general text issues are a minor part of functionality testing, some QA departments use dedicated localization testers. This is especially true with Japanese publishers. Early Japanese game translations were rife with Engrish, and in recent years this has become a more serious concern with Japanese developers as games have taken on a more cinematic presentation. To maintain the impact of the drama, localization testers are employed to review the game's script (which may be thousands of pages long for RPGs or other story-driven experiences), looking to make technical corrections as well as reviewing the translation work itself to ensure that it matches natural speech patterns. Thus, a localization tester acts as an editor. Though localization testing is currently a mainstay of Japanese game development, it will likely expand as the Chinese, Korean, and Russian game industries gain global traction. Those seeking work in this area do not necessarily need to speak foreign languages, but being able to understand the original texts is a helpful and desirable skill. Unlike other forms of testing, however, localization may involve very little actual game interaction. In some cases, the only times that the tester needs to play the game is to understand the context of dialog or interface messages that are not immediately obvious. The game's "script", which is not a script in the literary sense, but rather a collection of all the text strings referenced in the game, is typically catalogued in a spreadsheet that the tester is given to work with.
[edit] Soaking
Soaking, in the context of video games, typically involves leaving the game running for a period of hours or days in various modes of operation, such as idling, paused, or at the title screen. This test requires no user interaction beyond initial setup, and is therefore usually managed by lead testers. The purpose of soaking is to detect the presence of memory leaks or rounding errors that manifest over time, resulting in system instability or odd game behavior. Soak tests are one of the checked requirements for first-party compliance and must be performed on every title. Because it requires no user interaction, soaking is also a euphemism for employees shirking their duties (i.e. not playing the game).
[edit] Console hardware
For consoles, the vast majority of testing is not performed on a normal system (consumer unit). Special test equipment is provided to developers and publishers. Among these, the most significant pieces of hardware are the Test (or Debug) kits, and the Dev kits. The main difference between these special kits and consumer units is that, by default, they are able to load games from a burned disc or from a hard drive, as well as being able to set the console for any publishing region. Whereas this functionality is not present in consumer units in order to combat software piracy and grey-market imports, it is the only practical means by which game developers can produce copies for testing.[citation needed]
[edit] Test kits
A typical test kit has virtually the same hardware specifications and overall appearance as a consumer unit, though often with additional ports and connectors for other testing equipment. Test kits ordinarily contain a special system menu that allows the user to run automated compliance checks, especially with regard to save data. The system software also allows the user to capture memory dumps for aid in debugging.[citation needed]
[edit] Dev kits
Dev kits are not normally used by actual game testers, but are used by programmers for lower-level testing. In addition to all the features of a test kit, dev kits usually have higher hardware specifications, most notably more system memory. This allows developers to get an early look at the game without having to worry about optimizations. Dev kits also tend to be larger in size or have a different shape altogether than a test kit or consumer unit.[citation needed]
[edit] Methodology
There is no "gold standard" method for game testing, and most methodologies are developed in-house, by individual video game developers and publishers. And, methodologies are refined from time to time and may differ for different types of games (for example, the methodology to test a MMORPG will be different from the testing for a casual game). But outlined below is a typical methodology of game testing.
- Identification: Incorrect program behavior is analyzed and identified as a bug.
- Reporting: The bug is reported to the developers using a defect tracking system. At minimum, the circumstances of the bug and steps to reproduce are included in the report. Developers sometimes request additional documentation such as real-time video of the bug's manifestation.
- Analysis: This step of the process involves the developer responsible for the bug, such as an artist, programmer or game designer. It is outside the scope of the game tester's duties, though if there are inconsistencies in the bug report, the tester may be approached to provide more information or evidence.
- Verification: After the developer claims the bug to be fixed, it is the responsibility of the tester to verify that the bug no longer occurs. Not all bugs are addressed by the developer. Reported bugs may be claimed as features (often expressed as "NAB" or "not a bug"), and may also be waived (given permission to be ignored) by producers, game designers, or even lead testers, according to company policy.
- Regression testing: As game development progresses, closed bugs are checked again to ensure that they have not regressed, or re-manifested. For example, a programmer might undo a fix, having forgotten why the change was made in the first place. Though tedious, this type of testing is important since unexpected changes to code can cause old problems to resurface in the same or different ways.
- Standards testing: Consoles manufacturers such as Nintendo, Sony and Microsoft have standards they require to be met for games for their platforms. Not all manufacturers standards are alike and some testers are dedicated only to ensuring these standards are met.
[edit] See also
[edit] External links
- Sloperama: "Working as a Tester" from game industry veteran, Tom Sloper
- IGDA's Quality Assurance Special Interest Group
- National Academy of Video Game Testers and Reviewers (NAVGTR)
