A couple of weeks back I attended a local QA (Quality Assurance) conference in Columbus and I’d like to share some of the topics and insights presented.
#1 Testing Types
These are small tests written by a developer that check a small bit of code functions correctly. An example would be if a request to list all customers in fact returns all customers. There will
typically be a large number of these and they should cover the core of the website or application. These tests are run by a developer during the course of development and before each release. These can and should be setup to be highly automated.
Tests written by a developer that verify modules work correctly together. An example would be that when a user is submitted to the signup service all of the steps in the process complete
successfully (create the user in the database, added to the newsletter, and the welcome email is sent). These tests are run by a developer during the course of development and before each
release. These can can and should be setup to be highly automated.
Tests that check the website or application behaves in the expected manner from within a browser (or API), which includes page elements, interactivity/functionality, and error handling. An acceptance test could cover entering user login data, clicking on the login button, and checking for the result. These may be run manually, but should also be automated when possible. These are run by end users, project managers, and developers.
While manual tests may include acceptance tests, they also should be used to cover UI appearance, and link locations. Manual testing should be completed primary by the end client and project manager, but it’s recommended that all parties participate.
#2 Costs of Failure
The cost of failure gets more expensive as you work through the process (up the triangle), and as such, errors should be caught as early as possible. In an optimal scenario unit tests will cover all of the components, integration tests will test all systems, and acceptance tests will test all interfaces (or at least the user stories), leaving manual testing as an efficient final formality.
#3 Testing Environment
Web testing environments should mirror the production environment as much as possible, guaranteeing the
production environment will be sufficient. Because physical infrastructure can be costly to replicate, most web applications can be efficiently hosted on a virtualized platform, bringing full testing environments into reach for nearly all budgets.
Environments should use a copy of production data and connect to live APIs whenever possible.
#4 Maximize Automation
Whenever possible, opportunities to utilize automation should be taken to decrease testing time, reduce errors, and increase release timeliness. However, automation will take additional time to setup and may not be the best option for short, non-recurring projects. Having a fully automated test suite allows for agile projects, continuous integration, and generally a higher
confidence in quality.
#5 Whole Team Approach
Effective testing includes a whole team approach, shared information, straightforward results, and a commitment to quality. No one party will have full exposure to the entire lifecycle of the project, requiring testing from multiple parties.
Additional QA(ish) Testing
Tests the amount of concurrent traffic a website can handle before becoming unresponsive or unusable. This may also test that a site automatically scales correctly, if the platform is designed to automatically add servers during periods of high activity.
Tests the system for exploits that leak sensitive data or allow unauthorized access. Security tests can include third-party scans, penetration testing, and white-box security testing.
Disaster + Recovery
Tests that cover the procedures when the environment fails and the reliability of backups. If the environment is designed for high availability, this could test automatic or manual fail-overs for
each geographic zones.
Tests the load time for the website or application. These will focus on server response times, number of resources (images, scripts, etc) requested, and optimization techniques utilized.
Tests that accessibility criteria is addressed, which includes image alternate text data, proper headings, and other section 508 guidelines. Many commercial sites must properly handle
accessibility or be subject to litigation and/or heavy fines.