QA Library

Getting quality software out the door, on time, and in a functional, usable state is the great challenge of software development. Making sure the software does what it’s supposed to do, and does it in a user-friendly fashion is the difference between a successful product launch and something out of a Dilbert cartoon.

In the software development life cycle, quality assurance testing happens either after documentation is written (waterfall and iterative waterfall methods) or at the same time (agile methods). Each have their own unique challenges, but the end goal remains the same: releasing the most profitable software as quickly as possible.

The most elementary form of software testing is functionality testing; does the software, as written, enable the user to perform the job function it is meant to help with? While this can be obvious for some elements (a text editor that cannot save a file is clearly broken), for more obscure types of embedded software, this can be trickier to ascertain.

Likewise the more functionality a piece of software has, the more complex testing all of its variables in a reasonable time frame can become. To begin, work from both the technical specs and whatever nascent design documents exist, making sure that every part of the design specification is, in fact, met. Beware of any additional features that might have crept in during the development process. If this is multi-user software, remember to include load testing. This is always important in server and cloud computing environments.

An equally critical testing methodology for consumer-facing software is holistic, end-to-end user testing. It doesn’t matter how wonderful the software is if the consumer cannot figure out how to use it. This includes all end-to-end Black Box testing. It is astonishing what end-users will put software through, and one of the keys to successful quality assurance is the ability to identify any and all of it.

For certain products, once internal testing has reached a certain milestone, external testing begins. In these instances, Beta or Release Candidate software (depending on your testing methodology) is given out to a pool of real-world users. It is unlikely, at this phase, that any significant issues they find will be fit into the development schedule (due to the inherent risk involved in code changes at this stage). Issues found in this phase are usually triaged into three pools; those that are trivial to fix, those that fixing may break something else, and those that are effectively new feature requests for the next release.

One of the important aspects of external (beta) testing is that it helps form the initial wave of marketing support for the product. A recent example of this is how Windows 7 has gone through two Release Candidates and updated builds to emphasize new features in the OS, and to help build positive buzz. On the other hand, some companies (such as Google), keep functional software in an open beta, with incremental additions of features accreting over time, much as has happened with Google Apps.