Structure-based test design
techniques are
a good way of generating additional test cases that are different from existing
tests. They can help ensure more breadth of testing, in the sense that test
cases that achieve 100% coverage in any measure will be exercising all
parts of the software from the point of view of the items being covered.
Structure-based techniques
serve two purposes: test coverage measurement and structural test case design.
They are often used first to assess the amount of testing performed by tests derived
from specification-based techniques, i.e. to assess coverage. They are then
used to design additional tests with the aim of increasing the test coverage.
What is test coverage?
Test coverage measures in some
specific way the amount of testing performed by a set of tests (derived in some
other way, e.g. using specification-based techniques). Wherever we can count
things and can tell whether or not each of those things has been tested by some
test, then we can measure coverage. The basic coverage measure is
Number
of coverage items exercised
Coverage =
--------------------------------------------------- x 100%
Total number of coverage items
where the 'coverage item' is
whatever we have been able to count and see whether a test has exercised or
used this item.
There is danger in using a
coverage measure. 100% coverage does not mean 100% tested! Coverage
techniques measure only one dimension of a multi-dimensional concept. Two
different test cases may achieve exactly the same coverage but the input data
of one may find an error that the input data of the other doesn't. One drawback
of code coverage measurement is that it measures coverage
of what has been
written, i.e. the code itself; it cannot say anything about the software that has
not been written. If a specified function has not been implemented,
specification-based testing techniques will reveal this. If a function was
omitted from the specification, then experience-based techniques may find it.
But structure-based techniques can only look at a structure which is already
there.
How to measure coverage
For most practical purposes,
coverage measurement is something that requires tool support. However,
knowledge of the steps typically taken to measure coverage is useful in understanding
the relative merits of each technique. Our example assumes an intrusive
coverage measurement tool that alters the code by inserting instrumentation:
§ Decide on the structural
element to be used, i.e. the coverage items to be counted.
§ Count the structural elements
or items.
§ Instrument the code.
§ Run the tests for which
coverage measurement is required.
§ Using the output from the
instrumentation, determine the percentage of elements or items exercised.
Statement coverage
Studies and experience in the
industry have indicated that what is considered reasonably thorough black-box
testing may actually achieve only 60% to 75% statement coverage. Typical ad hoc
testing is likely to be around 30% - this leaves 70% of the statements
untested. Different coverage tools may work in slightly different ways, so they
may give different coverage figures for the same set of tests on the same code,
although at 100% coverage they should be the same. We will illustrate the
principles of coverage on code. In order to simplify our examples, we will use
a basic pseudo-code - this is not any specific programming language, but should
be readable and understandable to you, even if you have not done any
programming yourself. For example, consider code below,
READ A
READ B
I F A>B THEN C = 0
END IF
To achieve 100% statement
coverage of this code segment just one test case is required, one which ensures
that variable A contains a value that is greater than the value of variable B,
for example, A = 12 and B = 10. Note that here we are doing structural test design
first, since we are choosing our input values in order ensure statement
coverage.
Statement coverage is calculated by:
Number of statements exercised
Statement coverage =
--------------------------------------------- x 100%
Total number of statements
Decision coverage
A decision is an IF statement,
a loop control statement (e.g. DO-WHILE or REPEAT-UNTIL), or a CASE statement,
where there are two or more possible exits or outcomes from the statement. With
an IF statement, the exit can either be TRUE or FALSE, depending on the value
of the logical condition that comes after IF. With a loop control statement,
the outcome is either to perform the code within the loop or not - again a True
or False exit. Decision coverage is calculated by:
Number of decision outcomes exercised
Decision coverage =
-------------------------------------------------------- x 100%
Total number of decision outcomes
Condition Coverage
Here the coverage depends on
the testing of the Conditional Statements in the program. It mainly
concentrates on the Boolean conditional statements in the program. Eg. IF
(A=B&&A=C). it mainly concentrates on the various possible input
combinations.
Number of conditional outcomes
exercised
Condition coverage =
-------------------------------------------------------- x 100%
Total number of conditional
outcomes
