Introduction
Path-based testing
Get more detailed information about the current state of the art in path-based testing (process or workflow testing) to understand the benefits of using our proposed extension – Constrained Path-Based Testing
Why do we need Constrained Path-based Testing (CPT)?
Classical PBT might not be enough to handle several practical situations in system testing. To give few examples
Test cost management
Certain situations can be too costly for the given tests and have to be excluded from test scenarios.
Feasibility of certain tests
When a test reaches certain parts of the system workflow, internal state of the SUT might change and it disables continuation of the test in other certain part of the workflow.
Outages of particular parts of the test environment
In more complex projects, parts of the processes might be temporarily unavailable due to hardware or software limitations.
Testing of system component outages
Situations such as limited network connectivity, signal jamming or spoofing, or physical damage to system components have to be effectively tested from the process viewpoint.
CPT
Principle of the technique
Problem model
To maintain maximal compatibility with established path-based testing techniques, we base our model on directed graph, as usual in the field. In such model, nodes represent actions in the SUT and edges represent transitions between them.
Constraints
We extend the graph by a set of additional constraints, thorough which we can model number of practical testing situations that would be difficult to handle by classical path-based testing effectively.
Test Case Generation
Using the system model, our novel algorithms generate an optimized, cost-effective set of test cases that satisfy all the constraints specified in the input model. The principle of the algorithms is well documented in our research papers.
Application examples
Industrial application is the most important testing point telling us if the method is viable or not. We are successfully applying our methods to prepare tests for real systems. Here are a few examples of the projects in which we applied CPT:
DTA
Experimental sensor network monitoring vital signs of soldiers and increasing their survival chances in case of injury
TERESA
Telerehabilitation environment for patients recovering from post-acute COVID-19 disease
PATRAC
Rescue mission planning and management IoT system that helps mountain rescue service or state police to search for a missing person with a greater effectivity
Related works (under construction)
Number of previous works in a “classical” path-based testing exist. Few published studies also focused on the idea of constraints in path-based testing, however the concept was not systematically researched as a Model-based Testing subdiscipline, and some useful types of constraints were not examined at all yet. Here we attempt to create a reading list of relevant literature documenting a roadmap from “classical path based testing” to Constrained Path-based Testing (PT). If you find this list incomplete or inaccurate (which might easily happen, we would be grateful for suggestions how to extend it):
- Beizer, B. (2003). Software testing techniques. dreamtech Press.
- Ammann, P., & Offutt, J. (2017). Introduction to Software Testing Edition 2.
- Li, N., Li, F., & Offutt, J. (2012, April). Better algorithms to minimize the cost of test paths. In 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation (pp. 280-289). IEEE.
- Bures, M., Cerny, T., & Klima, M. (2017). Prioritized process test: More efficiency in testing of business processes and workflows. In Information Science and Applications 2017: ICISA 2017 8 (pp. 585-593). Springer Singapore.
- Miroslav Bures, Bestoun S. Ahmed, Employment of multiple algorithms for optimal path-based test selection strategy. In: Information and Software Technology, Volume 114, 2019, pp. 21-36.
- Miroslav Bures, Bestoun S. Ahmed and Kamal Z. Zamli.“Prioritized Process Test: An Alternative to Current Process Testing Strategies”. In International Journal of Software Engineering and Knowledge Engineering, 29(07), pp. 997-1028, 2019.
- Su, T., Wu, K., Miao, W., Pu, G., He, J., Chen, Y., & Su, Z. (2017). A survey on data-flow testing. ACM computing surveys (CSUR), 50(1), 1-35.
- Vivanti, Mattia et al. “Search-based data-flow test generation.” In: 2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE) (2013): 370-379.
- Girgis, M. R. (2005). Automatic test data generation for data flow testing using a genetic algorithm. J. Univers. Comput. Sci., 11(6), 898-915.
- Badlaney, J., Ghatol, R., & Jadhwani, R. (2006). An introduction to data-flow testing. North Carolina State University. Dept. of Computer Science.
- Klima, M., Bures, M., Ahmed, B. S., Bellekens, X., Atkinson, R., Tachtatzis, C., & Herout, P. (2023). Specialized path-based technique to test Internet of Things system functionality under limited network connectivity. Internet of Things, 22, 100706.
- Klima, M. (2023). Specialized Path-based Technique to Test loT System Behaviour under Limited Network Connectivity [Doctoral dissertation, Czech Technical University in Prague].
- Klima, M., Bures, M., Blaha, M., (2024) Ant Colony Optimization Based Algorithm for Test Path Generation Problem with Negative Constraints, In IEEE International Conference on Software Quality, Reliability and Security (QRS)
Free benchmarks and tool for everyone
We're introducing the CPT manager
We are developing a platform for the creation, management, and sharing of path-based testing SUT models. Let’s create a credible and balanced benchmark of models from real-life systems accompanied by generated problem instances, ensuring enough topology variety. Everyone can contribute. Let’s use this benchmark in further comparison of path-based testing algorithms generating the test cases. The environment supports the constraints specification inside the models and also offers automated test path generation for free!
