Effective testing for IT Services Software Projects

On this page:

• Introduction
• Code-Based Testing
• System Testing
• User Testing
• Functional Testing
• Usability Testing
• More Information

Introduction

Testing a software application should not be a haphazard activity with team members randomly inputting any value they can think of. Test cases should be planned so that all requirements and objectives are tested.

There are three different types of testing that need to be incorporated into any project that includes software development.

1. code-based testing
2. system testing
3. user testing

Use these questions to help you incorporate testing into the project plan:

• Have you planned for an overall testing schedule and the personnel required, and associated training requirements?
• Have you decided how you'll track test results (and resolve issues related to test failures)?
• Have you defined the test environment and tools needed for executing the software tests?
• Have you described the hardware configuration and resources needed to implement the designed test cases?
• Have you described the software configuration needed to implement the designed test cases?
• Have you described the way in which tests are to be recorded?
• Have you determined the criteria on which the completion of the test will be judged?
• Have you defined criteria for evaluating the test results?

The Three Types of Testing

Code-Based Testing

The first is Code-Based Testing. This is often accomplished though the use of unit tests, which are tests that are written into the software's code, by the developers. The best unit tests run automatically and can also be executed by non-developers, such as the Project Manager. Since code-based testing is different from other types of testing in that the person who writes the test will also be the one to run and check the results of the tests, it's important that the tests are reviewed by a third party. Having another developer review the unit tests is a good idea because the tests are generally written in the same code used to develop the software.

Questions to answer related to Code-Based Testing:

• What percentage of the testing is automated? (higher is better)
• Have you designed at least one test case for each system function?
• Have you designed test cases with special input values (e.g., empty files)?
• Have you designed test cases with default input values?
• Do all test cases agree with the specification of the function or requirement to be tested?
• Have you addressed error cases? Have you designed test cases for invalid and unexpected input conditions as well as valid conditions?
• Have you unambiguously defined test input data and expected test results or expected response messages for each test case?
• Have you documented the purpose of and capability demonstrated by each test case?
• Is it possible to meet and measure all test objectives defined?

System Testing

The second type is System Testing. These are tests deigned to determine how well the product performs on the system upon which it is running. System Testing is also concerned with testing the connections to and the dependencies on other systems. To execute well on your system testing, you'll need to have all the appropriate service providers involved early, so keep that in mind. Also, system tests should be run and evaluated by someone other than the system owner.

Questions to answer related to System Testing:

• Have you designed at least one test case for each system function?
• Have the test team members been given assignments?
• Have you designed test cases with special input values (e.g., empty files)?
• Have you designed test cases with default input values?
• Do all test cases agree with the specification of the function or requirement to be tested?
• Have you addressed error cases? Have you designed test cases for invalid and unexpected input conditions as well as valid conditions?
• Have you unambiguously defined test input data and expected test results or expected response messages for each test case?
• Have you documented the purpose of and capability demonstrated by each test case?
• Is it possible to meet and measure all test objectives defined?

User Testing

The third type is User Testing. There are actually two types of user testing. The first is *functional testing*. Functional tests are performed by users to determine if the software does everything the specification says it's supposed to. The second type of user testing is *usability testing*. Usability tests are used to discover problems related to ease-of-use. Test plans for functional tests should be jointly-written by the person responsible for spec (usually the Project Manager) and the person responsible for the user interface (usually the User Experience Specialist). Functional tests are usually completed by a single person sitting down and working through the test plan while taking notes. Usability tests require at least two people (though three or more is ideal). Three people is a good number because then you have one user, one moderator and at least one person to take notes and focus on observation. For all user tests, the authors of the test and the developers of the software are not considered good test subjects because they have knowledge about the software that typical users would not.

Questions to answer related to User Testing:

• Have you designed at least one test case for each system function?
• Have the test team members been given assignments?
• Have you designed test cases with special input values (e.g., empty files)?
• Have you designed test cases with default input values?
• Do all test cases agree with the specification of the function or requirement to be tested?
• Have you addressed error cases? Have you designed test cases for invalid and unexpected input conditions as well as valid conditions?
• Have you unambiguously defined test input data and expected test results or expected response messages for each test case?
• Have you documented the purpose of and capability demonstrated by each test case?
• Is it possible to meet and measure all test objectives defined?

Functional Testing

Functional Test Plan should include:

• Step-by-step directions on how to test each piece of functionality
• The desired result or behavior of each test
• Test results that *are not* desired should also be listed in the test plan, so testers can make sure those things don't happen.

Functional Test Environment:

• Testing should be performed on machines that are meant for testing (not the testers' primary machines).
• Test machines should be rebuilt after each test, preferably using a process that requires minimal user interaction (such as with a product like Ghost).
• Preferably, the testing lab/environment should be on a subnet that is isolated from the rest of the Stanford network. (Occasionally, catastrophic failures occur that affect the entire network, not just the machines being tested.)

Usability Testing

Usability Test Plan should include:

• An introduction of the moderator and anyone else in the room
• Disclaimer stating the test is voluntary and can be stopped at any time
• General information about what is being tested
• Demonstration of the testing protocol
• List of the areas of functionality to be tested
• Scenarios for why the user would be using the functional area being tested (if it's not already apparent)

Usability Testing Environment:

• In context testing is preferred. The software and services should be tested where ever users will be using them.
• Video is best medium for reporting test failures back to the project team. Consider recording all tests using a digital video capture device.