End-To-End Testing
What Is End-To-End Testing?
Due to the complexity of modern software and the unavoidable interconnections between different subsystems, the failure of any of these subsystems will cause the failure of the whole system. E2E testing is performed to avoid this scenario, as it’s designed for verifying a system flow as the whole organism. The purpose of E2E testing is to check the application flow from the very start to an end, imitating a complete real-world scenario. E2E testing aims to check the system dependencies and to ensure the correct integrity of different parts of the system as a whole. During E2E testing the whole system is tested for critical functionalities such as databases, networks, and communication with other systems or apps. Also known as chain testing, it takes production like data and simulates real-life scenarios. The testing is done mainly after functional and system testing, helping to detect problems within sub-systems and ensure their integration as a complete system. E2E testing validates how the software functions at every level, on various systems, and as a complete organism, proving that end-user experience works as intended. E2E testing also means more detected bugs, as in the Agile development environment is usually updated with every iteration or in every few iterations, which ensures bugs being caught and corrected faster, instead of migrating through the whole development process and described at the end of the process. E2E testing also means more detected bugs, as in the Agile development environment is usually updated with every iteration or in every few iterations, which ensures bugs being caught and corrected faster, instead of migrating through the whole development process and described at the end of the process.
Consequently, E2E testing helps to reduce costs and test time spent on testing. The basic process of E2E testing consists of the following steps:
Requirements planning - during this phase, the analysis of the existing system/subsystems, and recommended software and hardware requirements is done.
E2E System Design - during this phase, planning of the component tools, workflows, individual roles, and responsibilities is performed.
Testing Component Design - for each subsystem suites of test cases are written.
Test Infrastructure Development - during this phase hardware and software infrastructure is set up according to the test environment.
Test Execution and Measurement - the final phase is for testing the system, recording input and output data, analyzing and reporting all the flows and failures.
There are 2 main methods of E2E testing — horizontally and vertically.
Horizontal E2E TestingThe horizontal method of E2E testing means that testing is done from the user’s perspective, running through each phase from the start to end, ensuring each connected process correct execution.
Vertical E2E TestingVertical method of E2E testing intends testing each layer of a system from top to bottom, testing each subsystem independently, then testing is continued through the network infrastructure, API integration, and UI layers.
Although both methods of E2E testing help to ensure the entire system correct functionality, each of them has pros and cons. The main advantages of horizontal E2E testing are focused on the end-user perspective to ensure the system’s correct work, proper coverage of business logic requirements, independent set up and execution, higher coverage of core code, and the ability to be implemented in stages, without setting up the full application test environment. The main drawbacks of horizontal E2E testing are slowness of test execution and bug fixing and the necessity of coordination by the main SDLC stakeholders. One of the vertical E2E testing drawbacks is also the necessity of coordination by the main SDLC stakeholders. The main advantages of vertical E2E testing are higher code coverage fast test execution and bug fixing, and the ability of implementation in stages. Three main stages of E2E testing are user functions (building functions that imitate real user behavior), test conditions (creating conditions to evaluate passed or failed tests), and test cases (building test cases that execute functions according to the chosen scenario and evaluate performance against the test conditions).
Time management is vital in any testing process, and E2E testing is not an exception. Consequently, there are key metrics for the successful E2E testing process. Such metrics are:
● Test case preparation status, which includes the comparison of test cases that are prepared or are in the process of preparation, and those that were planned.
● Weekly testing progress, which includes the evaluation of the weekly completed tests as a percentage from total planned, along with the numbers of passed and failed tests.
● Bug status and details, which includes the percentage of opened and closed bugs (weekly), based on priority and severity.
● Test environment availability, which includes evaluation of the time spent on running tests.
Now, let us define what the E2E testing basic implementation rules are.
There are 4 of them:
1. Identification of the most important tests is the first one. For large companies’ applications, E2E testing can become massive in scope. It would be tiresome to test every written test case. Instead, it is reasonable to choose the most relevant test cases, or the ones prone to defects and run the most valuable tests within the app.
2. Choosing a testing framework is also an important activity, as the evaluation of different frameworks for finding the most suitable one, is essential for the business. Here are the most popular E2E testing frameworks:
● Protractor
● Nightwatch
● CasperJS
● Testcafe
3. The third major E2E testing implementation activity is the integration of E2E testing with existing SDLC. While the most significant tests are identified, and the testing framework is chosen, it is logical to understand how designed E2E testing can be implemented into existing SDLC.
4. The final activity is creating a prototype testing environment and process for validating a new testing system without interrupting regular workflows, or possibly disrupt existing codebase. During this activity, an isolated repository of code is created. Then comes running tests in a staging environment. When the testing process-perfected, integration with existing SDLC and infrastructure is performed.
To sum up, E2E testing plays an essential role in testing the application’s overall functionality and behavior over connected systems.
Benefits:
● Ensures Complete Correctness & Health of the Application
● Increases Confidence in the Application
● Test main paths in end to end perspective
● Uncover integration defects
● Test the alternative paths for the same objective
● Test from the business perspective
