What is Automation Testing?

Do you want to learn automation testing but don’t know how to begin? Let us guide you through the basics of automation testing along with the whole test automation process.

In this tutorial, we will study – what, how, when, and why of automation testing. During this tutorial, we will get to know some of the most popular test automation tools. You can pick any of these tools for learning, based on your skill-set and the project requirements.

Skip to our tutorials on automation testing tools directly.

Selenium	Katalon	Cucumber	TestNG

What is Automation?

Before starting with Automation Testing, let’s first understand the term – “automation”. Automation is a process using which we can automate a manual process with the use of technology. The aim is to eliminate or reduce human/manual effort. Now let’s see, how automation helps in software testing.

Testing software includes writing test cases once and then executing them again and again, whenever required.

The test execution, when performed manually is a very time-consuming and tedious task. Automation testing helps in reducing the test execution time. As the test scripts written once can be run automatically any number of times without any human intervention.

What is Automation Testing?

Automation testing is a type of software testing that involves automated test case execution using an automation tool.

So, basically, it automates the manual testing process. The tester writes test scripts and then runs the test scripts either on-demand or schedule them for periodic executions. This reduces the overall testing time, thus helping in faster product releases.

What to Automate?

Now that we know what exactly is Automation testing, let’s check which test cases to automate. Or, what all test cases are ideal candidates for automation.

1. Test cases that test critical functionality of the application

For example for an e-commerce application, the critical functionality would be the product discovery via search and category pages, add to cart and then buy functionality. So, these test cases should be first chosen. Test cases for add to wishlist and notify me etc should be of lower priority. Hence should be picked accordingly for automation.

2. Test cases that require repeated test execution with a large dataset

There are many test cases or application flow that require performing an action again and again. Such test cases are also ideal candidates for automation as automation reduces a considerable amount of testing effort.

Take an example of a search feature of an application. If we can automate the flow to search with a search term and then verify that search results. Then we can run the same script again and again with different types of search terms like – single word, multi-word, alphanumeric, with special characters, with foreign language characters, etc.

3. Tests that are time-consuming

Workflows that require a considerable amount of time to execute and set up are also ideal candidates for automation.

Let’s take the example of e-commerce application forward. If some test cases require the set up of multiple products and then performing some operation on those products. Such test cases when automated not only reduce the test execution time but also, free the manual testers of the redundant task. In addition. helping them focus on other exploratory testing activities.

4. Test cases that require parallel execution

There are scenarios that require checking the concurrent access to the application e.g. in the case of performance testing with multiple users. In such cases, either manual testing is not feasible or it would require a lot more resources to test the particular scenarios. In those cases, automated scripts help by creating concurrent requests and collating results in one place.

What not to Automate?

It is also important to understand, what sort of test cases cannot or rather should not be automated.

1. UI test cases

GUI or the Graphical User Interface test cases should best be left for manual testing or human validation. This is because even with the slightest change in the UI, the test cases would fail. Moreover, it is also very difficult to create reliable UI test cases across multiple devices and screen resolutions.

2. Usability test cases

Rather than ‘should not’ it is the case of ‘cannot’ automate. Usability test cases, test application’s ease of use by different sets of users which, with the current technology is not possible to automate.

3. Functionalities that are rarely used and take time for scripting

It is good to automate complex scenarios but investing your effort in scenarios that would rarely be used doesn’t provide a good Return on Investment.

4. Exploratory testing

Exploratory testing requires learning about the application on the fly and simultaneous testing. Hence, it is not possible to automate exploratory testing scenarios.

When should we Automate?

After defining, all the capabilities of the automation suite during Test Planning, we can begin the automation framework creation activity in parallel with the development team. But the scripting of the test cases should be done at the right time.

For better automation ROI and to avoid any rework – scripting of test cases should start when the application is stable and frequent changes in the application are not anticipated.

Why Automated Testing?

Following are some of the advantages or benefits of automation testing-

• Automation testing reduces the overall test execution time. Since automated test execution is faster than manual test execution.
  
  
• It reduces the cost and resource requirements of the project. This is because the script created once can be made to run any number of times as long as there is no change in the application.
  
  
• Helps in working with a large set of input which isn’t be feasible with manual testing.
  
  
• Helps in creating a Continuous Integration environment wherein, after each code push, automatic test suite execution takes place with the new build. Using CICD tools like Jenkins, we can create Jobs that execute the test cases after the deployment of a build and mails the test results to the stakeholders.
  
  
  

When not to Automate?

Let’s see some scenarios, where it is not advisable to do automated testing along with some disadvantages of automation.

• Lack of expertise of the automation tool – Lacking expertise in the automation tool and\or programming language to create robust scripts is one of the primary reason which can lead to not using the tool to its full potential. Factors like these lead to the failure of automation testing.
  
  
• Incorrectly chosen test cases – The success of automation testing heavily depends on the test cases chosen for automation. Incorrectly chosen tests lead to wastage of resources and time invested in automation.
  
  
• Applications with frequent changes – Choosing test automation for an application with frequent changes requires constant maintenance of the test scripts, which at times might not give the desired ROI.
  
  
• Inefficiently written test scripts – Test scripts with limited or inadequate validations can lead to false-positive test results. These false-positive results conceal the underlying defects which could have been easily captured if validated manually or scripted in a better way.
  
  
  

Test Automation Process

The automated testing process includes all the set of activities that are performed during the automation of different software applications. We will learn each phase from the requirement understanding to the automated scripting and the CI-CD integration phase.

1. Requirements understanding

Before starting with test automation, the first and foremost activity is to understand the requirement. The understanding of the requirement will help in defining the scope of automation along with a selection of the right tool.

2. Defining scope of automation

Defining the scope of automation is finding the right test cases for automation. This would include all the types of test cases that fall under the test case types defined in the “What to Automate?” section of this article.

3. Selecting the right tool

Tool selection depends on various factors like – the requirement of the project, the programming expertise, project budget (whether to go for the free or paid tool), etc.

4. Framework creation 

For creating robust test automation suites we need to create an automation framework. These frameworks help in making the test scripts reusable, maintainable, and robust. Based on the project requirement, we can choose among the different automation frameworks discussed below, in the section – Automation Testing Framework.

5. Scripting test cases

After the automation framework set up, we start the scripting of test cases, selected for automation. A typical script for a web application test case looks like this-

• Open browser
• Navigate to the application URL
• Perform some actions on different web elements
• Post some data picked from external test data files
• Validation or assertion logic
  
  

6. CICD Integration

Although we can run the test cases on demand but nowadays CICD – (Continuous integration and continuous delivery) is widely used in almost every product or service-based company. From a testing point of view, this involves setting up the test suites on a CICD tool like Jenkins.

Once set up, we can then schedule a nightly test suite run or run on test cases on demand. Another major benefit of integrating the automation suite with the CICD pipeline comes when we can trigger the test cases automatically for execution after the deployment. In this setup, in just one click, just after the deployment, the automation test suites check the stability of the build.

Test Automation Framework

While doing manual testing of any application, we define a set of rules. For example – the format of the test cases, the prioritization of test execution, the different practices for optimizing the whole software testing process, etc. Similarly in the case of automation testing also, we make use of automation frameworks that help in reducing the automated script maintenance cost. Also, improving the overall automation testing process.

The framework can contain guidelines and implementation of different factors that affect the quality of the test suite like- 

• Where should we have the test data? – Hard-coded in the script or in some external files.
• How should we handle the object repository or the locators of the elements be automated (locators are the identifier of the different elements like – a textbox, a button, etc.)?
• Where should we keep the common functionalities that can be reused again and again? – To avoid redundancy and improve code-reusability.
• What should be the reporting format? – HTML, PDF, Xls, etc.
• How should we implement logging? – What tool/library should be used? 
• What should be done in case of a test failure? – Should we take a screenshot or try to re-run the test case, etc?
  

To handle and implement all such practices, we develop an automation testing framework. Following are some of the most widely used automation frameworks-

Modular framework

Modular frameworks are one of the basic automation frameworks in which the commonly used functionalities database connection, login flow, method to read data from an external file, etc are identified and created as methods. While scripting instead of writing the line of code, again and again, we simply call the reusable methods wherever required.

Data-driven framework

In the case of data-driven frameworks, the test data lies in external files, and based on the number of rows on the external files, the test case gets executed multiple times with different datasets in each iteration (fetched from the file). Since the data is driving the automation, hence the name, Data-driven framework.

Keyword-driven framework

In a keyword-driven framework, we can write test cases in plain text. For example, we can write test cases in an excel sheet in the below form-


The framework will already have the methods defined for each keyword used, like – there will be automation code for OpenBrowser keyword, NavigateTo keyword, etc. In this way, once the framework is set up even a non-technical user can create automation test scripts in plain text.

Hybrid framework

As the name suggests, it is the combination of multiple frameworks. So, a combination of any 2 of the above framework would be termed a data-driven framework. Mostly when we say a Hybrid framework it relates to a Data-driven with a keyword-driven framework.

For more details, check – Test Automation Frameworks.

Myths about Automated Testing

Time to bust some myths surrounding test automation-

• 100% automation is possible – Barring a few very specific applications, 100% automation is just not possible. Exploratory test cases, usability test cases are some of the examples of test cases that can’t be automated.
  
  
• Automation will eat the manual testing jobs – It is true that with the advent and rise of automation testing and different tools, there is a shift to the requirement of full-stack or dual role testers – who can work on both manual and automation testing. But automation will never be able to completely eliminate the need for manual testing.
  
  
• Developer makes better automation testers – While a developer might have a slight edge on the coding part but still a tester will be able to think from a testing perspective.
  
  
• Automation is expensive – When done correctly, automation can reduce the overall testing effort and resource requirement. Thus saving the project cost in the long run.
  
  
  

Popular Test Automation Testing Tools

The software market is full of paid and free test automation tools. Based on the various factors like – project requirement, budget, the expertise of the resources, etc, we should choose the right tool, suited for our needs. The following are some of the most popular test automation tools in the market.

Selenium

Selenium is an open-source test automation tool. It has a very large and active community. It has the maximum market share among all the popular tools and supports scripting in multiple languages – Java, Python, Ruby, Javascript, C#, etc. Download link – Selenium Download

Katalon Studio

A fairly new tool but rapidly getting popular due to record and playback features along with scripting for more technical users. It is free but not open source.

UFT One

A paid tool by Microfocus that can be used for automation of both Web and Windows applications. It supports scripting in VBScript only.

TestComplete

A paid tool provided by Smartbear can be used for automation of Web, Mobile as well as Desktop applications.

Tosca

It is a paid tool by Tricentis that provides record and playback features for automating web applications, APIs, and windows applications. It is considered one of the most popular codeless automation tools that can completely eliminate the need for scripting.

Watir

Watir (Web Application Testing In Ruby) is an open-source automation tool that can automate web applications in Ruby. It has a watir-webdriver component that is based on Selenium.

Appium

It can be considered as Selenium for mobile applications. Just like Selenium, it is open-source and has a large user base.

TestProject

TestProject is a free and community-powered automation testing tool by Tricentis. It can be used for the automation of both web and mobile applications.

Ranorex

A paid tool with record and playback features. Using this we can automate android, ios and Windows applications. Along with record and playback, it also supports scripting in using C# and VB scripts.

For more details, you can check our post – Top Test Automation Tools containing in-depth review, tool’s features, download link, and our recommendations of the different tools.

Our Tutorials on Automation Testing Tools

Now, that you have a fair understanding of the test automation process, you can select the right tool for learning automation. For this, I would highly recommend you to go through the post – Roadmap for transition from Manual to Automation QA. After that, you can check our below-mentioned tutorials on popular test automation tools.

Selenium	Katalon	Cucumber	TestNG

Conclusion

I hope this article helped you in understanding the automation testing basics. Also, you would have understood the different activities performed in the test automation process.