Once you've completed your prototype solution, you need to complete some robust and comprehensive testing to ensure that it really does work. You will of course have carried out iterative testing as you went along, but some final testing just before handover would also be important to do. Note that many of your tests may show that things don't quite work as they should. This is an important part of the process. Make a note of the things that don't work, see if you can fix them, and comment on their functionality and what you could improve.
Testing involves testing the program under various conditions to make sure it is going to work. You need to think about different types and possible user input that could be tried on the system and what might cause the program to crash.
Developers use test plans that check a range of types of data under different circumstances.
A test plan is a list of requirements designed to ensure that the coded solution works as expected. The test plan will include specific instructions about the data and conditions the program will be tested with.
We can use a test plan to check the following section of code to see if there are any errors.
first = input("Input the first number")
second = input("Input the second number")
print("The sum is",first+second)A simplified test plan might look like the following table:
| Feature | Test cases | Expected result | Actual result |
|---|---|---|---|
| Addition of whole numbers | 1st: 0, 4, 44, 4444 and 2nd: 0, 4, 44, 4444 | 0, 8, 88, 8888 | 00, 44, 4444, 44444444 |
| Addition of decimals | 1st: 1.0, 1.2, .01 and 2nd: 1.0, 1.2, .01 | 2.0, 2.4, 0.02 | 1.01.0, 1.21.2, .01.01 |
| Handling of strings | 1st: one, two and 2nd: one, two | Syntax error because it can't add strings together | oneone, twotwo |
The 'Actual result' column is giving some unexpected results. The computer program executes without any syntax errors so there must be a semantic error.
To fix this semantic error, you would need to convert both values into an integer. One solution would be to perform an in-line integer conversion on each of the first two lines. For example:
first = int(input("Input the first number"))
Before performing a test, you need to decide what data you are going to include in your test case. It is not normally possible to perform tests with every single possible piece of data. So, instead the developers will choose from a limited range of data such as:
Tests should find that the program works as expected. Obvious input data should confirm that the software works as expected. Extreme test data will be chosen to test what breaks the system.
For example, if you were developing a number-guessing game, you might have a unit of code that asks the user to choose a number in a specific range, eg "Choose a number between 1 and 10". To test this unit, you could try a whole range of inputs to see what happens:
3, 4.5, three, -99, 10.00001
Sometimes software developers use something called an invariantwhen they are testing or fixing bugs. An invariant is a value or condition that can be relied upon to be true when a program is being executed.
Using something that can be relied upon helps the developer to isolate the units that are working from the units that are not.
When testing more complex examples of software, it is sometimes necessary to test a number of conditions and sub-conditions at the same time.
A trace table, also called a trace matrix, can be used to record the outcomes of the test. The trace table for a very simple example, such as x=y+2, would look like this:
| y | x |
|---|---|
| 13 | 15 |
| 25 | 27 |
| 1200 | 1202 |
A trace table might appear to look very similar to a test plan, but without the other headings like 'expected outcome' and 'actual outcome'.
Alpha testing relates to the first round of testing. Beta testing refers to the second round of testing.
Before releasing new software on to the market, developers will test it in-house. This is alpha testing.
The developers will then trial the software with a limited number of users. The users report any problems to the developers. This is called beta testing. Once the developers are confident that the software works, they will release it for sale or distribution.
The final test is when the product is released and real end users begin using it. Very often, end users find things wrong with systems that the programmers did not expect.
Updates are often released for operating systems, applications and games because customers have found software bugs. Reporting errors online to the developers allows them to investigate what has gone wrong. The developers can then fix the problems and release updates.
Write your own summary of the problem. What are your objectives? List the success criteria
| Test No. | Description | Test Data(input) | Expected Outcome | Actual Outcome | Further Action? |
|---|---|---|---|---|---|
| 1 | |||||
| 2 | |||||
| 3 | |||||
| 4 | |||||
| 5 |