Why do we need Constrained Path-based Testing (CPT)?

Why do we need Constrained Path-based Testing (CPT)?

Classical PT might not be enough to handle several practical situations in system testing. To give few examples:

Test cost management. Certain situations can be too costly for the given tests and have to be excluded from test scenarios. 

Feasibility of certain tests. When a test reaches certain parts of the system workflow, internal state of the SUT might change and it disables continuation of the test in other certain part of the workflow. Such a situation in the industry is usually resolved “ad hoc when it happens”, in the test design, or in the test execution. However, such approach will be very likely sub-optimal from cost viewpoint. 

Outages of particular parts of the test environment. In more complex projects, parts of the processes might be temporarily unavailable due to hardware or software limitations. This fact has to be also reflected in the tests – it’s faster than letting 20 testers spend a few hours to discover that a certain part of SUT in the test environment is not working and it’s not just a random set of bugs. How many MDs in total will we effectively lose by such an outage? 4? 8? Quite enough to motivate us to find a systematic solution for this, isn’t it? Good test environment monitoring and management can help here and reduce these losses. When further powered up by good test case management, even better. 

Time consuming asynchronous operation. When action A represents the start of an asynchronous operation and action B represents its end, the rest of the system can be tested in the meantime. In this case, such a test case that would call again the operation between actions A and B must not be present in the exercised set of the test scenarios.

A need to use only a reduced subset of resources. There might be processes in a SUT that use some resources extensively and when the availability of such resources is limited, they need not to be executed. For example, an energy intensive process might not be executed on a system with low battery level.

What to do in such situations, having a classical path-based testing toolset only?

Few intuitive solutions are available:

  1.  “Twist the reality” when modeling a SUT process and model it in the way that test cases that are not possible will not be generated. Honestly, this is the wrong approach and will very probably lead to confusion later. In test automation, one of the golden rules is “Don’t change SUT because your tests would need it to overcome some obstacle. At least do it if you really don’t have other options”. We can rephrase it to “Don’t twist the SUT model to sort out a limit of an MBT method”. One of the biggest challenges in MBT (and probably one of its biggest drawbacks) is how to keep the model up-to-date with the design documentation or/and with the SUT. So don’t add to this problem.
  2. Exclude test paths (from the generated test set) that would violate the given condition. However, there are drawbacks: (1) it might take time, (2) you can make a mistake in this process, and (3) you reduce the test set perhaps more than it is needed to solve the problem in a more systematic way.