Modified condition/decision coverage

Modified condition/decision coverage (MC/DC) is a code coverage criterion used in software testing.

YouTube Encyclopedic

1/3
Views:
61 685
31 805
2 527

Transcription

And this criterion is called Modified Condition/Decision Coverage, also called MC/DC. This criterion is very important because it is often required for safety critical applications. For example, the FAA, the Federal Aviation Administration, requires for all the software that runs on commercial airplanes to be tested according to the Modified Condition/Decision Coverage. So what is the key idea behind the MC/DC criterion? It is to test only the important combinations of conditions instead of all of them, and limit the testing cost by excluding the other combinations. And the way in which it works is by extending branch and decision coverage with the requirement that each condition should affect the decision outcome independently. So let's see what this means with an example that will also show you how you can reduce the number of combinations in this way. I am going to show you an example of how MC/DC works using this predicate which consists of three conditions. a, b, and c, which are all in and, so the overall predicate is a and b and c. The first thing I'm going to do is to show you how many test cases we will need to satisfy the multiple condition coverage for this simple predicate. Which means, how many test cases we will need to test all the possible combinations of true and false values for these conditions. So I'm going to populate this table. And as you can see, at the end we have eight test cases. Each test case tests a different combination of values for a, b, and c. I'm also showing, for each test case, the outcome of the overall predicate. So, for example, if we look at the first one, the first test case, will be such that a is true, b is true, and c is true. And therefore, the overall outcome of the predicate is true. Now lets consider the first condition, a. As I said a minute ago, what we want to test are the important combination. Which are the comibatinos in which a single condition independently effects the outcome of the overall predicate. So if we consider a and we look at this possible set of this cases. Let's try to find two test cases such that the only difference between the two test cases is the value of a, and the overall outcome of the predicate is different. If we look at the table, we can see that this is true for test cases one and five. If we look at these two cases, we can see that the overall of the predicate in the two cases is true and false, and that the only difference between the value of the conditions in the value of a. So these test cases satisfy exactly what we wanted. There are two test cases in which the value of a independently decides the overall value of the predicate. What we do, therefore, is to add these first two test cases to our set of tests down here. Now let's focus on b and let's try to find two test cases such that the value of b is the only value that changes between the two test cases, but the overall value of the predicate is different, the same thing we did for a. And in this case, we can see that if we select test case number one, and test case number three, we have exactly that situation. b is true in the first case, false in the second one, a and c don't change, but the overall value of the predicate changes. And now you can notice something else. That even though we selected two test cases, tested two values, one we already had. So, we only need three test cases overall to test a and b according to MC/DC. Now, let's look at our last condition, c. At this point, we know the game, so we just have to look for two test cases that satisfy our requirements. And in this case, one and two are suitable candidates. And once more, because we already have one, we just have to add two to our list. So as you can see from this example, we went from having eight test cases needed to cover all the possible combinations of conditions to only four test cases to satisfy the MC/DC criteria. So let's see where MC/DC stands in our substantion hierarchy. This is what we had so far in the hierarchy and this is where the MC/DC criterion will stand. MC/DC criterion is stronger than branch and condition coverage. Why? Because it requires every single condition to be true and false. And therefore, this section was a condition coverage criteria. And it also requires every predicate to be true and false and therefore, this section is branch coverage. And in addition, it's got the additional requirements that the true and false values, all the conditions have to also decide the overall value of the predicate. So it's stronger. Which is more thorough than branch and condition coverage and, as usual, also stronger than branch coverage, statement coverage, and condition coverage.

Overview

MC/DC requires all of the below during testing:^[1]

Each entry and exit point is invoked
Each decision takes every possible outcome
Each condition in a decision takes every possible outcome
Each condition in a decision is shown to independently affect the outcome of the decision.

Independence of a condition is shown by proving that only one condition changes at a time.

MC/DC is used in avionics software development guidance DO-178B and DO-178C to ensure adequate testing of the most critical (Level A) software, which is defined as that software which could provide (or prevent failure of) continued safe flight and landing of an aircraft. It is also highly recommended for SIL 4 in part 3 Annex B of the basic safety publication^[2] and ASIL D in part 6 of automotive standard ISO 26262.^[3]

Additionally, NASA requires 100% MC/DC coverage for any safety critical software component in Section 3.7.4 of NPR 7150.2D.^[4]

Definitions

Condition: A condition is a leaf-level Boolean expression (it cannot be broken down into simpler Boolean expressions).
Decision: A Boolean expression composed of conditions and zero or more Boolean operators. A decision without a Boolean operator is a condition. A decision does not imply a change of control flow, e.g. an assignment of a boolean expression to a variable is a decision for MC/DC.
Condition coverage: Every condition in a decision in the program has taken all possible outcomes at least once.
Decision coverage: Every point of entry and exit in the program has been invoked at least once, and every decision in the program has taken all possible outcomes at least once.
Condition/decision coverage: Every point of entry and exit in the program has been invoked at least once, every condition in a decision in the program has taken all possible outcomes at least once, and every decision in the program has taken all possible outcomes at least once.
Modified condition/decision coverage: Every point of entry and exit in the program has been invoked at least once, every condition in a decision in the program has taken all possible outcomes at least once, and each condition has been shown to affect that decision outcome independently. A condition is shown to affect a decision's outcome independently by varying just that condition while holding fixed all other possible conditions. The condition/decision criterion does not guarantee the coverage of all conditions in the module because in many test cases, some conditions of a decision are masked by the other conditions. Using the modified condition/decision criterion, each condition must be shown to be able to act on the decision outcome by itself, everything else being held fixed. The MC/DC criterion is thus much stronger than the condition/decision coverage.

Criticism

It is a misunderstanding that by purely syntactic rearrangements of decisions (breaking them into several independently evaluated conditions using temporary variables, the values of which are then used in the decision) which do not change the semantics of a program can lower the difficulty of obtaining complete MC/DC coverage.^[5]

This is because MC/DC is driven by the program syntax. However, this kind of "cheating" can be done to simplify expressions, not simply to avoid MC/DC complexities. For example, assignment of the number of days in a month (excluding leap years) could be achieved by using either a switch statement or by using a table with an enumeration value as an index. The number of tests required based on the source code could be considerably different depending upon the coverage required, although semantically we would want to test both approaches with a minimum number of tests.^{[citation needed]}

Another example that could be considered as "cheating" to achieve higher MC/DC is:

/* Function A */

void function_a (int a, bool b, bool c, bool d, bool e, bool f)
{
    if (a == 100)
    {
        if (b || c)
            // statement 1
        
        if (d || e || f)
            // statement 2
    }
}

/* Function B */

void function_b (int a, bool b, bool c, bool d, bool e, bool f)
{
    bool a_is_equal_to_100  = a == 100 ;
    bool b_or_c             = b || c ;
    bool d_or_e_or_f        = d || e || f ;

    if (a_is_equal_to_100)
    {
        if (b_or_c)
            // statement 1
            
        if (d_or_e_or_f)
            // statement 2
    }
}

if the definition of a decision is treated as if it is a boolean expression that changes the control flow of the program (the text in brackets in an 'if' statement) then one may think that Function B is likely to have higher MC/DC than Function A for a given set of test cases (easier to test because it needs less tests to achieve 100% MC/DC coverage), even though functionally both are the same.^[6]

However, what is wrong in the previous statement is the definition of decision. A decision includes 'any' boolean expression, even for assignments to variables. In this case, the three assignments should be treated as a decision for MC/DC purposes and therefore the changed code needs exactly the same tests and number of tests to achieve MC/DC than the first one. Some code coverage tools do not use this strict interpretation of a decision and may produce false positives (reporting 100% code coverage when indeed this is not the case). ^{[citation needed]}

RC/DC

In 2002 Sergiy Vilkomir proposed reinforced condition/decision coverage (RC/DC) as a stronger version of the MC/DC coverage criterion that is suitable for safety-critical systems.^[7]^[8]

Jonathan Bowen and his co-author analyzed several variants of MC/DC and RC/DC and concluded that at least some MC/DC variants have superior coverage over RC/DC.^[9]

References

^ Hayhurst, Kelly; Veerhusen, Dan; Chilenski, John; Rierson, Leanna (May 2001). "A Practical Tutorial on Modified Condition/ Decision Coverage" (PDF). NASA.
^ IEC 61508-3:2010
^ ISO 26262-2011 Part 6 Table 12
^ "NASA Software Engineering Requirements". NASA.
^ Rajan, Ajitha; Heimdahl, Mats; Whalen, Michael (March 2003). "The Effect of Program and Model Structure on MC⁄DC Test Adequacy Coverage" (PDF). {{cite journal}}: Cite journal requires |journal= (help)
^ http://www.hbni.ac.in/phdthesis/engg/ENGG02201004005.pdf ^{[bare URL PDF]}
^ Vilkomir, S.A.; Bowen, J.P. (2002). "Reinforced condition/decision coverage (RC/DC): A new criterion for software testing". International Conference of B and Z Users. Lecture Notes in Computer Science. 2272. Springer-Verlag: 291–308. doi:10.1007/3-540-45648-1_15. ISBN 978-3-540-43166-4.
^ Vilkomir, S.A.; Bowen, J.P. (2006). "From MC/DC to RC/DC: formalization and analysis of control-flow testing criteria". Formal Aspects of Computing. 18 (1). Springer Nature: 42–62. doi:10.1007/s00165-005-0084-7. S2CID 10467796.
^ Kapoor, Kalpesh; Bowen, Jonathan P (2005). "A formal analysis of MCDC and RCDC test criteria". Software Testing, Verification and Reliability. 15 (1). Wiley Online Library: 21–40. doi:10.1002/stvr.306. S2CID 35276126.

External links

This page was last edited on 15 May 2024, at 06:04

From Wikipedia, the free encyclopedia