System Testing

Software testing is an activity to uncover potential errors and bugs in software applications before releasing them to end users. It takes place in four different levels – unit testing, integration testing, system testing, and acceptance testing. In this article, we will learn about system testing in detail. 

But before that, knowing about the different software testing levels is essential. 

• Unit testing – It involves testing each individual component of software separately to ensure that it works correctly. Generally, developers or programmers perform unit testing during the development phase of SDLC. 
• Integration testing – The next comes integration testing. It logically combines unit-tested components into groups and verifies their interaction. It uncovers errors that may occur due to the interaction of software modules. 
• System testing – Once the whole system is combined and integrated into one, it is evaluated as a whole. The purpose is to check the system from the user’s point of view. 
• Acceptance testing – The final level is acceptance testing. A small group of real users or an internal team of an organization checks whether a software product fulfills all the specified requirements. 

Let’s now move on to understand system testing in detail.

What is System Testing?

System testing is a type of software testing that evaluates a software product as a whole against functional and non-functional requirements. It determines the overall performance and functionality of a fully integrated software product. 

The primary goal of this testing type is to check that all software components work together without any flaws and function as intended while meeting all the specified requirements. It is concerned with verifying the software product’s design, behavior, and compliance with customer requirements. 

A QA team carries out system testing after the integration testing and before acceptance testing. They choose a testing environment that closely resembles the actual production environment. Since the QA team tests the entire system without knowing its internal workings, it falls under black-box testing. 

Integrated modules that have passed integration testing serve as the input to system testing. Integration testing uncovers defects or irregularities between the integrated units. However, system testing discovers defects between integrated units and the whole system.

In a nutshell, this software testing type involves performing a series of tests to exercise the entire software.

 

System Testing Example

Let us take a real-world example to understand this. Consider the car manufacturing process. Initially, a car manufacturer produces all the essential components, such as brakes, an engine, seats, steering, wheels, etc. After manufacturing these components, it’s time to test them individually, which we call unit testing in software development. 

Once the functionality of all these individual components is confirmed, the manufacturer assembles them together. The next step is to check whether the assembled combination does not result in any error or has no side effects on the functionality of each component. We refer to this as integration testing. 

After ensuring no defects between the assembled combination, the manufacturer checks this combination as a whole, which is system testing. The car as a whole undergoes multiple checks to verify it meets the specified requirements, like the car running smoothly, all other components (brakes, gears, wheels, etc.) working correctly, etc. 

When the car meets customers’ expectations, they are more likely to buy it. 

Why do we Need System Testing in Software Testing?

• Even after successful unit and integration testing, many complex scenarios may have undiscovered issues. System testing helps in uncovering those defects.
• It tests the software against functional and non-functional requirements. This happens for the first time in the entire software development life cycle. Hence, it verifies the software’s architecture or design and business requirements. 
• The testing environment closely matches the production environment. Hence, successful system testing brings a sense of confidence in the final delivered product. Also, stakeholders can understand how end users react to the software.
• It minimizes post-deployment issues, troubleshooting, and support calls. 
  
  

What to Verify in System Testing? 

This testing type assesses the software product for the following: 

• The interaction between software components, including external peripherals, to verify how the software works as a whole. This is the scenario of end-to-end testing. 
• Inputs given to the software produce the expected outcomes. 
• Functional and non-functional requirements are met. 
• End users’ experience with the software. 

We have listed some of the most important parameters here. However, system testing involves validating many other aspects. It requires detailed test cases and test suites to the software’s every aspect from the outside, without peeking into its internal details. 

 

Entry and Exit Criteria 

Each software testing level has entry and exit criteria. The following are the entry and exit criteria for system testing: 

Entry Criteria 

• The software should pass all the exit criteria of integration testing, i.e., the execution of all test cases should be finished, and there should be no critical or priority bug in an open state. 
• The test plan should be approved and signed off. 
• Test cases, test scenarios, and test scripts should be ready for execution. 
• The testing environment should be ready. 
• Non-functional requirements should be at hand. 

Exit Criteria 

• All the intended test cases for system testing should be executed. 
• No priority or critical bug should be in an open state. 
• Even if medium or low-priority bugs are open, they should pass on to the next level of testing – acceptance testing. 
• The exit report should be ready. 
  
  

System Testing Types 

As stated earlier, This testing type evaluates the software for functional and non-functional requirements. Hence, the software must undergo various testing techniques to assess the whole system and its different aspects.

The different types of system testing are – 

1. Functional testing: It assesses the software to check whether each functionality works as intended and meets the specified requirements. If testers find some functionalities missing, they list them for the development team for implementation. Also, they suggest additional functionality to enhance the software. 
2. Performance testing: It is a non-functional testing type that validates the system for stability, scalability, responsiveness, and speed under the given load.  
3. Usability testing: It is also a non-functional testing that checks the user-friendliness and effectiveness of the software. Simply put, it determines how easily users can manage and operate the software and access its features. 
4. Reliability testing: This testing technique evaluates the software to check whether it functions correctly and consistently under a specific condition for a given period.  
5. Security testing: It discovers all security risks and vulnerabilities in the software and ensures that it does not allow any unauthorized access to data and other resources. 
6. Scalability testing: This type of load testing evaluates the software for its performance when the number of users scales up and down. 
7. Recoverability testing: This testing technique determines the software’s ability to recover from failures and crashes. 
8. Interoperability testing: It analyzes how well the software interacts with its components and third-party products. 
9. Regression testing: It ensures that any new changes in the software’s source code do not affect the existing functionality. It assures the stability of the software as it integrates subsystems and maintenance procedures. 
   
   

System Testing Process

Here are the different steps of the system testing: 

1. Setting Test Environment – The first step is to create a test environment that matches the production environment for quality testing. A test environment includes selecting programming languages, frameworks, and testing tools and establishing necessary dependencies and configurations. 
2. Creating Test Cases – The next step is to create test cases for an exhaustive testing process. It also involves creating a test document containing the count of passed and failed test cases. 
3. Developing Test Data – This step involves collecting test data. It should include all necessary information and fields. Identify favorable and unfavorable input/output combinations. 
4. Executing Test Cases – Use the created test cases and test data to execute them. This helps you know whether the test cases are successful or unsuccessful. 
5. Detecting Bugs/Defects – If any bugs or errors are encountered, testers should be reported in the test document created in the second step. 
6. Regression Testing – To fix the encountered errors, developers make changes to the source code. So, testes perform regression testing to ensure that the most recent modifications to the source code do not affect its existing functionality.
7. Retest – If errors are found during regression testing, the testing team reports them to the development team. The testing cycle continues until the software is ready to go into the production stage. 
   
   

Advantages and Disadvantages of System Testing

Here are some significant advantages and disadvantages of system testing: 

Advantages 

• System testing does not require testers to have programming knowledge. 
• It validates the entire software product and uncovers errors and defects that unit and integration testing cannot. 
• The testing environment is similar to the actual production environment. 
• The thorough testing of the software product ultimately results in high quality. 
• It improves the overall performance, maintenance, and reliability of the system. 
• As it uncovers all possible bugs and errors, the development and testing teams are confident enough to release products to users.
  
  

Disadvantages 

• System testing is time-consuming.
• It requires highly skilled testers.
• It is challenging for large and complex projects. 
• Testers do not have visibility into the software’s source code. 
• No testing uncovers 100% bugs. Even if system testing validates every aspect of the source code, bugs may still exist.
  
  

System Testing vs Acceptance Testing 

The following table highlights the difference between system testing and acceptance testing: 

Conclusion

This was all about system testing in software engineering. It is a crucial level in software testing, as it thoroughly tests software products against functional and non-functional requirements. If done properly, it improves the software’s quality, performance, maintainability, and reliability. Otherwise, the possible errors that were left undiscovered may pop up when the software becomes live.