Adding constraints to the system model
Number of practical testing situations and special types of tests can be effectively handled by using constraints added to the system model.
Examples of these constraints are:
- If you do an action A, you must do an action B
- If you do an action A in a test, you can’t do action B
- If you do an action A in a test, you can do action B, but only once
And other suitable types. If a situation or a special path-based testing technique cannot be handled by a set of such constraints, it’s good reason to add other type. Otherwise let’s not do it until it’s necessary, to keep the complexity of the problem low.
More detail and a formal description
We define a constraint as an attribute of a pair of graph elements (we selected to start with nodes) that together either must or must not be present in the set of test cases. More formally, constraint c is a pair of SUT model nodes; let’s call them A and B. These nodes represent actions in the SUT process that influence one another. We define two types of constraints in our model.
Positive constraints
Positive constraints serve to model situations in which we need to follow a particular action with another action in the test. For example, we need to test the boundaries of a subpart of the process, entering and leaving this region by defined entry and exit points, not just randomly. An example of a situation when we need to do this is testing of component outage situations in an IoT system.
Positive constraints (C) are pairs of specific graph elements that must be present in at least one test case in the final test set. The order of the nodes must remain as specified in the constraint, meaning that node A must precede node B in a test case.
Relation of test coverage criterion and positive constraints
We are searching for a set of test paths containing all pairs of positive constraints defined in the model in at least one test path. Moreover, the SUT model’s edges must be present at least once in the test set. Therefore, we can say that apart from satisfying the positive constraints, test set T also satisfies the edge coverage (or TDL=1) of the SUT model.
Negative constraints
In real-life systems we often exercise a scenario in which executing a specific element restrict another element in the process. Such a situation is not sufficiently covered in the current process testing practice, and therefore we introduce negative constraints.
Negative constraints (NC) can be of two types. The EXCLUSIVE type of constraint means that the first node in the pair, A, must not be followed by node B in any test case in the final test set. The ONLY_ONCE type of constraint means that node A can be followed by node B only once in the final test set.
Relation of test coverage criterion and negative constraints
While maximizing edge coverage in the SUT model, we are searching for a set of test paths that respects all negative constraints defined in the model. For this reason, for certain situations, defined negative constraints might make 100% edge coverage or higher coverage levels impossible in a set of test paths.