A comparative analysis of API testing approaches in continuous integration/continuous deployment (CI/CD) pipelines in cloud based systems

Abstract

This study examines how a multi-layer testing framework, which is made up of Contract Testing, Integration Testing, and End-to-End (E2E) Testing, can be used to guarantee the reliability and robustness of RESTful APIs. The main aim was to test the contribution of each of the testing layers toward the identification of structural, logical, and communication level flaws, in the API-based system. The case study is a Flask-based booking API, in which seven control defects were introduced systematically to test the detection efficiency and execution performance at each of the testing phases. The experimental results showed that Contract Testing had the lowest execution time (55 seconds) but had the lowest Defect Detection rate (10 percent), with schema and structural mismatches in the API contracts being the most frequent. Integration Testing took more time (155 seconds) and had the best DDR (80 percent), which is successful in the detection of most interface, data-flow, and cross-endpoint behavioral defects. End-to-End (E2E) Testing was the slowest (230 seconds), and the DDR of 56.7%; however, it was realistic in validating entire user processes and business rules and revealed some logic and process-level anomalies that had not been exposed by Contract Testing. Overall, the aggregated values of the execution time and the DDR allow stating that a layered testing approach, which involves contract, integration, and E2E testing, is a decent trade-off between speed, coverage depth, and reliability of API-based architectures.