Endurance Testing – Soak Testing

What is Endurance Testing?

Endurance testing is a testing of the software to check system performance under specific load conditions over an extended or longer amount of time. Endurance testing is a non-functional testing and it falls under types of Performance testing.

It is also known as Soak testing and it aims at testing the system for a prolonged period and make sure that the system runs smoothly without any failure and handles the desired load. In endurance testing, the testing is conducted for a longer period as per the testing requirements (such as 12 hours, 72 hours or more) compared to other testing types such as load or stress testing.

Endurance testing helps in finding out any memory leaks in the system. It also helps in checking the response time of the system over a longer period.

Example of Endurance Testing

Let’s try to understand the importance of endurance testing by taking an example of the ‘Memory leak’ concept.

A memory leak is a concept used in computer science. The simple definition of memory leak is “a failure to release objects that are no longer used due to a weak programming code written by a programmer”. In the event of memory leak, due to memory used by unused objects, available memory reduces gradually which slows down the system and eventually the system stops working.

In endurance testing, testers run the system for a predetermined amount of time by providing a certain amount of load to check for any memory leak.

Handling memory leaks completely depends on the programming language used.

Another real-world example can be of a bank. Let’s say a strike is announced and banks will not work for straight 3-4 days. During this period, online banking activities will increase as compared to other normal business days. The number of users and the number of operations will increase as well. Due to this, a system needs to be ready to handle such load.

Endurance Testing Process

Performing an endurance testing process requires detailed planning as such test runs require to last for a longer period. Let’s take a look at steps involved in endurance testing process-

  • Business and System Requirements Analysis – In this step, the testing team studies the business and system requirements to identify the need for endurance testing. If the system is an already tested existing system and only new features are added, then endurance testing might not be required. But if the system is a newly developed one, endurance testing becomes a necessary action.

  • Test Planning – In this step, once the endurance testing need is identified, the testing team starts preparing the test plan. Resources required for the test run and, cost and time estimation for the complete testing process is finalized. The extent of testing methods, manual and automation, and automation tools to perform endurance testing (if used) are identified. Moreover, hardware, software, database, and network requirements are finalized in this step which should stimulate the actual production environment. 

  • Designing Test Scenarios and Test Cases – Once the detailed project plan is outlined, the testing team works on designing test scenarios and test cases for the testing process. Also, during this step, the breakpoint of the system is identified i.e. the amount of load that needs to be used for the testing process.

  • Test Environment Set up – In this step, the test environment is set up and it is advisable to keep it separate from the actual production environment and other test environments as well.

  • Test Execution – Now, once the environment is ready, test cases are executed and if there are any issues identified during this step, they are noted down. Developers are responsible for solving them and retesting is performed once they are fixed.

  • Closure – Once the test cycle satisfies the exit criteria for the testing process, it will be closed, and a report of identified issues and their final status will be prepared.

Advantages of Endurance Testing

This section discusses the advantages of endurance testing as given below-

  • When the system runs for a longer period, it might face performance-related problems. Endurance testing helps in finding these issues and checks the software’s durability.
  • Endurance testing helps in finding bugs related to a memory leak. 
  • Endurance testing gives confidence to the development team and the client that the system will run smoothly for a longer period without any maintenance, especially during long weekends. 
  • Like other performance testing types, endurance testing also results in customer satisfaction. Performance issues arising once the system is available with the users would result in reputation damage.
  • Another advantage of endurance testing is a reduction in maintenance cost which can be higher if certain performance-related issues are not found during the testing cycle.
  • Faults and bugs found during endurance testing help the development team and the client to improve the infrastructure.

Disadvantages of Endurance Testing

The following are some disadvantages of endurance testing-

  • It is a time-consuming process as it might go on for 12 hours to 12 months.
  • There should be proper research on whether endurance testing is required or not. For example, any website which does not generate much traffic or deal with sensitive data, and for which regular maintenance is available, endurance testing may not be a priority. 
  • Endurance testing is a costly process as it requires the system to run continuously throughout the test run which will also require suitable infrastructure.
  • It is important to keep the endurance testing environment separate from other test environments as it might cause application failures or data loss.
  • Endurance testing requires proper planning especially when it comes to deciding on the number of hours the test run(s) needs to be run.
  • Manual endurance testing is not possible so an automation tool will have to used to conduct the testing. The testing team should be skilled in the automation tool(s) to be used.

Endurance Testing Tools

The following are some tools used for endurance testing-

  • JMeter – Apache JMeter is a performance testing tool written in JAVA. It is open-source software, freely available and platform-independent. Moreover, it supports integration with Selenium, it can also be used as a unit testing tool.
  • LoadRunner – Like JMeter, LoadRunner is also a performance testing tool and, can be used as a unit and integration testing tool. It is a market leader when it comes to performance testing. LoadRunner also supports scripts from JMeter and Selenium by declaring an interface library. It is not free but allows a certain number of users in the trial version. 
  • Appvance – Appvance is an automation tool used for functional, performance and security testing among others. It is an AI-driven tool that provides a virtual user dashboard and real-time analytics.
  • OpenSTA – It is an open-source tool and used to perform scripted HTTP and HTTPS heavy load tests with performance measures. OpenSTA stands for Open System Testing Architecture. It is written in C++ by CYRANO and supports only Microsoft Windows operating systems.
  • WebLOAD Professional – Another performance testing tool that has various pricing plans, and like NeoLoad it comes with a free plan with a limited set of users. It also supports Selenium and Perfecto Mobile.

Conclusion

Endurance testing or Soak testing is a type of performance testing that is used to observe system behavior when the system runs for a longer period with a specific load. It is a time-consuming process and demands proper planning. It is useful in identifying performance relates issues that might arise due to the continuous running of the application.

Leave a Comment