Home Topic 1 Last Next


Suggest various types of testing.


Teaching Note:

The crucial importance of testing at all stages of implementation should be emphasized, with the stages clearly defined.

Types of testing can include: user acceptance testing, debugging, beta testing.

Students should be aware that there are programs that can test other programs, thereby automating parts of the testing process and reducing costs.

S/E Inadequate testing can reduce employee productivity and lead to end-user dissatisfaction.

Sample Question (former curriculum):

Identify three different types of data that should be included as part of a testing strategy. [3 marks]


Sample Question (former curriculum):

Outline a method that can be used to test a program. [2 marks]

JSR Notes:

Firstly, it is important to remember that we are dealing with Systems here throughout Topic 1, so this assessment statement should ultimately be focused whole testing of systems, not just software testing, even that's a good place to start. But do recognize that various stages of software testing are just part of the overall system testing.

Testing is crucial at all stages of IT system implementation. Testing needs to be done continually throughout the entire development process. Whole departments of IT companies specialize in testing. By helping to catch errors in the development process, the result is a more robust and reliable end product. Without adequate testing, a problematic or inefficient product can result in reduced employee productivity and user dissatisfaction.


General Testing Considerations/Strategies:

- Test that all of the outputs are as expected.

- Test for completeness, and check against the criteria for success.

- Assure the use of actual intended users to test it (not just the developer who knows the program and how to use it inside and out.)

- Make sure you use a complete data set for testing, including for normal, abnormal, border cases, and extremes. For example, if testing a grading program, here are examples of each category of data (grades in this case):

Normal 90
Abnormal 110
Border cases 0 and 100
Extreme 23423424332
Wrong type ninety

- Netbeans and other more sophisticated software development suites have automatic ways to test and debug software applications. These applications are able to come up with appropriate data sets, and run that data through the program with the click of a button. This can reduce costs, particularly of big projects where there may be entire departments of people whose sole job it is to test software. (This is referred to in the teaching note.)

In the case of the hardware components of the system, there are analogous considerations and strategies that must be followed for thorough testing before the system is rolled out for real. We don't usually talk about "bugs" in hardware, but there are many, many potential issues and limits which must be dealt with.

An idea of the scope of testing over the development of a system

Software Testing and Debugging

Software testing is carried out in order to increase the reliability and quality of a program by:


Debugging is the process of finding and removing/solving bugs, i.e. the errors, in a program. So, this is typically regarded as a software stage of testing. Though the term debugging may be used for addressing all kinds of bugs/problems, be they software or hardware as well.

There are three general kinds of software errors you need to check for, and debug.

Kinds of Errors

- Syntax Error, i.e. the "spelling" and "grammar" of the programming language is incorrect. Your IDE (Integrated Development Environment, such as Netbeans, IntelliJ etc.) does syntax checking automatically for you, either as you type, or when you compile the program. Syntax errors are typically indicated with red squigglies under the problematic text, and often also a rollover messages indicating what the syntax error likely is.

Common syntax errors (and their messages) are things like:

; expected      when you forget the ; at the end of a line
cannot resolve symbol SCcaner      when you mistype an identifier, such as "SCcaner" for "Scanner"
'else' without 'if'

     when you forget a { or }

- Semantic Error - Semantic errors are errors in logic. By logic, we mean the way that the program will run based on the instructions you are giving it. So semantic errors have nothing to do with spelling/grammar mistakes; rather they are errors in the way you form the logic, which the computer is to follow to accomplish something. So when program runs, it will not accomplish what you had intended.

Netbeans and other IDEs will generally not catch logic errors, and so even after compilation, there will be no red squigglies, and the program will run and execute your error in logic. A simple example of a logic error is where, for the calculation of an average, you add up 6 numbers and divide by 5 - even if there's no syntax error in your code, it is logically wrong.

- Runtime Errors - These are errors that occur while you are running the program, which cause the program to freeze or to stop functioning properly. Different programming languages have different approaches to dealing with the possibility of runtime errors. Java has specific classes made for handling what are termed runtime "exceptions", and later on you will learn how to use these to protect your programs from freezing. Some common exceptions you will code your programs to be ready to "catch" include:

(You could say it is semantic errors that often cause runtime errors, but we tend to think of semantic errors as errors in our code that we should notice before runtime, and runtime errors as the actual particular error that occurs during the running program, regardless of its cause.)


Testing to Eliminate All Categories of Errors

Syntax errors are mostly picked up by the IDE when writing code, so this is not so much a concern of testing stages. During testing, it's mainly semantic errors that will be picked up, through running a wide battery of tests. And also, if/when runtime errors are encountered during testing, steps can be taken to prevent them happening again.



A/B Testing 

This form of testing involves gathering feedback from users about a specific kind of change to a system. One group of users, Group A, is shown the original version of the system, and another group, Group B, is shown a new version of the system. Using feedback, the developer can determine whether or not it is a good idea to switch to the new system.



User Acceptance Testing

With user acceptance testing the product is tested by client community against the requirements provided as a final check before the product is released in beta form.

This is the kind of testing you will employ throughout the development of your IA solution, via interviews and interactions with your client. Does your client like your program? Can they use it without your help? Is it what they wanted? And ultimately, overall, do they accept the software (or system) that has been developed for them?

For your IA, an overall client acceptance of "yes" or "no" can be determined via testing which thoroughly covers the list of initial expectations of the client, which will be in the "criteria for success" document.

In the real world of IT and business, the criteria for success will have been worked into a specific document associated with the contract, and in the case of differing interpretations or conclusions, lawyers will regularly be involved.

In the case of the hardware components, the user must also accept their quality, quantity, and functioning for the IT system to be considered properly implemented.




Beta Testing

This is the releasing of a trial version into the real world so it can be tested under real-world conditions by users who are willing to give an early version a try. Beta testing should be done only after initial user acceptance testing. The "beta" version ("beta" is the second letter of the Greek alphabet) is not expected to be a completely finished and polished ("alpha") product. So both the company releasing the product, and the users themselves should expect there will be imperfections and small problems.

Often it will be the more geeky individuals out there that are willing to try out the imperfect beta software since they are excited to see and use the new product, they don't need it to be perfect, and they are happy to help the developer by providing feedback. Their experimentation with software before the release of a final version helps the developer greatly, particularly when the feedback is regular, and even immediate. Feedback is canvased automatically particularly when the program freezes. The user will be prompted to send a report, which is generated, detailing when and under what circumstances the program stopped working properly.

Hardware is obviously harder to test in a "beta" way, but certainly this happens also. A good example recently is various incarnations of 3d printers which, version by version, get better and better, but the newest is alway bound to have a "glitch" or "bug" or two. Never-the-less, the consumers of the latest "beta" are happy to risk potential problems, by having the latest technology at their fingertips.




Specific Example of Testing a System

Examples of various software and hardware in the proposed Hospital Database system:

- Testing the network systems by overloading the servers

- Testing the id cards - does overuse cause damage, what happens if they are cut in half or damaged in come way

- Thorough testing of all IT equipment in the ambulances and other first responder vehicles

- Testing of the various database system applications

This is just a small sample of the kinds and varieties of testing which would have to be carried out. Proper testing for a large IT system like this is a huge endeavor carried out, by many people, continually over the life cycle of the development. Note that organizing testing into stages and groups of tests (as seen above) is often referred to as Unit Testing.

Referring to the teaching note above, recognize that a major project of an IT company would be the planing and scheduling and assigning of the thousands of tests that would be required to make sure this system worked in all sorts of conditions. You can imagine charts and tables of all the various stages of testing needed to be carried out, the people and teams and indeed companies responsible for the various testing, the dates the testing needs to be completed by, and the next steps upon failed tests.

In this case, the consequences of the system not being absolutely thoroughly tested is the difference between life and death for the patients served by the system.




WikiBooks.org **

Jamie: Unit testing vs. integration (i.e. with the other components to test interactions)