Test Coverage Levels
Function coverage
|
→ Which functions of your code have already been called?
|
Function coverage measures during tests:
- how many times each function has been entered;
- how many times its closing brace has been reached ( ≈ how many times function's code has been fully read).
This is a quite weak test coverage measure but gives anyway a rough estimate for the thoroughness of the testing.
|
Decision (Branch) coverage
|
→ Which decisions in your code (if-statement or loop conditions) have already been evaluated both to "true" AND "false"?
|
For each function, Decision coverage measures whether:
- each branch in a switch-statement has been taken or not;
- each explicit control transfer (goto, break, continue, return, throw) has been taken or not;
- each exception handler has been visited.
This code coverage metric enables you, regarding Function coverage, to get a more accurate percentage of already run code.
|
Condition coverage
|
→ Which conditions of your code (e.g. boolean expressions separated with a logical "or" in an if-statement) have already been evaluated both to "true" AND "false"?
|
Looking a bit like Decision coverage, Condition coverage measures, for each conditionnal node of each function, whether all atomic boolean sub-expressions contained in the condition have already been evaluated both to "true" and "false".
|
Modified condition/decision coverage - MC/DC - MCDC
|
→ Which conditions in your code have already been evaluated both to "true" AND "false" affecting independently the decision's outcome?
|
On top of Condition and Decision coverage criteria, Modified condition/decision coverage implies that each condition should be evaluated (at least one time) in a case where it affects independently the decision's outcome. A condition is shown to independently affect a decision's outcome by varying just that condition while holding fixed all other possible conditions.
This metric conforms to the international technical standard DO-178C which specifies the criteria for software certification for mission-critical equipment and systems within the aviation industry .
- Pros: needs less test cases than MCC
( n conditions ⇒ n+1 tests).
- Cons: test cases will not always be easily found...
|
Multicondition coverage - Multiple condition coverage - MCC
|
→ Which decisions of your code have already been evaluated with all possible combinations of conditions?
|
On top of Condition coverage criteria, Multicondition coverage implies that, for each decision, each possible combination of boolean values for conditions should be evaluated.
- Pros: an assured code quality since all possibilities are tested.
- Cons: needs more test cases than MCDC.
( n conditions ⇒ 2n tests).
|
The Test Coverage Analyzer Testwell CTC++ can analyses for all kinds of test coverage criteria (particularly for MC/DC and MCC) requested for the development of critical software in order to get certifications.
Glossary:
Condition: a logical indivisible (atomic) expression. It is often called boolean variable, represented by a capital letter (A, B, C, etc.), can only be equal to "true" or "false", but can not be divided in other simpler "sub-conditions".
Decision : a logical expression which can be composed of several conditions separated by logical operators like "or", "and", "xor".
Papers:Grundlagen der Code Coverage Messung
Videos auf deutsch:Grundlagen der Code Coverage Messung (103)
Code-Coverage-Stufen (104)
MC/DC und Multicondition Coverage (105)
|