Do you want to learn automation testing but don't know how to begin? Let us guide you through the basics of automation testing along with the whole test automation process.
In this tutorial, we will study - what, how, when and why of automation testing. During this tutorial, we will get to know some of the most popular test automation tools that you can pick for learning based on your skill-set and project requirements.
What is Automation?
Before starting with Automation Testing, let's first understand the term - automation. Automation is a process using which we can automate a manual process with the use of technology. The aim is to eliminate or reduce human/manual effort. Now let’s see how automation helps in software testing.
Testing a software includes writing test cases once and then executing them again and again, whenever required. The test execution, when performed manually is a very time consuming and tedious task. Automation testing helps in reducing the test execution time as the test scripts written once, can be run automatically any number of times without any human intervention.
What is Automation Testing?
Automation testing is a type of software testing that involves automated test case execution using an automation tool.
The tester writes test scripts and then run the test scripts either on-demand or schedule them for periodic executions. This reduces the overall testing time, thus helping in faster product releases.
What to Automate?
Now that we know what exactly is Automation testing, let’s check which test cases should be chosen for automation.
- Test cases that test critical functionality of the application - For example for an e-commerce application, the critical functionality would be the product discovery via search and category pages, add to cart and then buy functionality. So, these test cases should be first chosen. Test cases for add to wishlist and notify me etc should be of lower priority and hence picked accordingly for automation.
- Test cases that require repeated test execution with a large dataset - There are many test cases or application flow that require performing an action again and again. Such test cases are also ideal candidates for automation as once automated, considerable amount of testing effort gets reduced.
Take an example of search feature of an application. If we can automate the flow to search with a search term and then verify that search results. Then the same script can be run again and again with different type of search terms like - single word, multi-word, alphanumeric, with special characters, with foreign language characters etc.
- Tests that are time consuming - Work flows that require considerable amount of time to execute and setup should also be ideal candidates for automation.
Taking the example of e-commerce application forward, if some test cases require the set up of multiple products and then performing some operation on those products. Such test case when automated not only reduce the test execution time but also, free the manual testers of the redundant task and help them focus on other exploratory testing activities.
- Test cases that are required to be run in parallel or distributed environment - Some test cases that require performing of concurrent operations on the application e.g. in case of performance testing or scenarios in which we need to check application behavior on simultaneous access to the resources. In such cases, either manual testing is not feasible or it would require a lot more resources to test the particular scenarios. In those automated scripts help by making concurrent requests and collating results at one place.
What not to Automate?
It is also important to understand, what sort of test cases cannot or rather should not be automated.
- UI related test cases - Test cases related to UI should best be left for manual testing or human validation. This is because even with the slightest change in the UI the test cases would fail and it also very difficult to create reliable UI test cases across multiple devices and screen resolutions.
- Usability related test cases - Rather than 'should not' its the case of 'cannot' automate. Usability related test cases tests application’s ease of use by a multiple set of users which with the current technology is not possible to automate.
- Functionalities that are rarely used and take time for scripting - It is good to automate the complex scenarios but investing your effort on scenarios that would rarely be used doesn’t provide a good Return on Investment.
- Exploratory testing - Exploratory testing requires learning about the application on the fly and simultaneous testing. Hence, it is not possible to automate exploratory testing scenarios.
When should we Automate?
Once all the capabilities of the Automation Suites are defined during Test Planning, we can begin the automation framework creation activity in parallel with development team. But the scripting of the test cases should be done at the right time.
For better automation ROI and to avoid any rework - scripting of test cases should be started when the application is stable and frequent changes in the application are not anticipated.
Why to Automate?
Advantages of Automation Testing
- Automation testing reduces the overall test execution time. Since, automated test execution is faster than manual test execution.
- It reduces the cost and resource requirements of the project, as the script created once can be made to run any number of times as long as there is no change in the application.
- Helps in working with large set of input which isn't be feasible with manual testing.
- Helps in creating Continuous Integration environment where in, after each code push, automatic test suite execution takes place with the new build. Using CICD tools like Jenkins, we can create Jobs which execute the test cases after the deployment of a build and mails the test results to the stakeholders.
When not to Automate?
Let's see some scenarios, where it is not advisable to do automated testing along with some disadvantages of automation.
- Lack of expertise of the automation tool - Lacking in the use of automation tool and\or programming language required to create robust scripts is one of the primary reason which can lead taking too much time for test script creation or not using the tool to its full potential. Factors like these leads to failure of automation testing.
- Incorrectly choosen test cases - Success of automation testing heavily depends on the test cases chosen for automation. Incorrectly chosen tests leads to wastage of resources and time invested in automation.
- Applications with frequent changes - Chosing test automation for an application with frequent changes requires constant maintainence of the test scripts, which at times might not give the desired ROI.
- Inefficiently written test scripts - Test scripts with limited or inadequate validations can lead to false positive test results. These false positive results conceal the underlying defects which could have been easily captured if validated manually or scripted in a better way.
Test Automation Process
Let’s see all the activities that are performed during the test automation process right from right gathering till the automated test case execution.
- Requirements understanding - Before starting with test automation, the first and foremost activity is to understand the requirement. The understanding of the requirement will help in defining the scope of automation along with selection of the right tool.
- Defining scope of automation - Defining scope of automation is finding the right test cases for automation. This would include all the type of test cases that fall under the test case types defined in the “What to Automate?” section of this article.
- Selecting the right tool - Tool section depends on various factors like - the requirement of the project, the programming expertise, project budget (whether to go for free or paid tool) etc.
- Framework creation - For creating robust test automation suites we need to create automation framework. These frameworks helps in making the test scripts reusable, maintainable and robust. Some of the popular frameworks in automation are-
- Modular framework - Modular frameworks are one of the basic automation frameworks in which the commonly used functionalities database connection, login flow, method to read data from external file etc are identified and created as methods. While scripting instead of writing the line of code again and again, we simply call the reusable methods wherever required.
- Data driven framework - In case of data driven frameworks, the test data lies in external files and based on the number of rows on the external files, the test case gets executed multiple times with different dataset in each iteration (fetched from the file). Since, the data is driving the automation, hence the name, Data driven framework.
- Keyword driven framework - In keyword driven framework, we can write test cases in plain text e.g. we can write test cases in an excel sheet in the below form-
Now the framework, will already have the methods defined for each keyword used like - there will be automation code for OpenBrowser keyword, NavigateTo keyword etc. In this way, once the framework is setup even a non-technical user can create automation test scripts in plain-text.
- Hybrid framework - As the name suggests, it is the combination of multiple frameworks. So, a combination of any 2 of the above framework would be termed as data driven framework. Mostly when we say a Hybrid framework it relates to a Data driven driven with keyword driven framework.
- Scripting test cases - After the automation framework setup, we start the scripting of test cases, selected for automation. A typical script for a web application test case looks like this-
- Open browser
- Navigate to the application URL
- Perform some actions on different web elements
- Post some data picked from external test data files
- Validation or assertion logic
- CICD Integration - Although we can run the test cases on demand but now a days CICD - (Continuous integration and continuous delivery) is widely used in almost every product or service based companies. From testing point of view, this involves setting up of the test suites on a CICD tool like Jenkins.
Once setup, we can then schedule nightly test suite run or on test cases on demand. But the major benefit of integrating the automation suite with CICD pipeline comes when the test cases are automatically triggered for execution after the deployment. In this setup, in just one click, just after the deployment, the automation test suites checks the stability of the build.
Myths about Automated Testing
Time to bust some myths surrounding test automation-
- 100% automation is possible - Barring a few very specific applications, 100% automation is just not possible. Exploratory test cases, usability test cases are some of the examples of the test cases that can’t be automated.
- Automation will eat the manual testing jobs - While it is true that with the advent and rise of automation testing and different tools, there is a shift to the requirement of full stack or dual role testers - who can work on both manual and automation testing. But automation will never be able to completely eliminate the need for manual testing.
- Developer make better automation testers - While developer might have a slight edge on the coding part but still a tester will be able to think from testing perspective and try to create more robust test scripts that will have multi-gate verification.
- Automation is expensive - When done correctly, automation can reduce the overall testing effort and resource requirement. Thus saving the project cost in the long run.
Popular Test Automation Tools
- Selenium - Being free, open source with a large community, Selenium is one of the most popular browser automation tools. Download link - Selenium Download
- Katalon Studio - A fairly new tool but rapidly getting popular due to record and plaback features along with scripting for more technical users. It is free but not open source.
- QTP/UFT - A paid tool that can be used for automation of both Web and Windows applications.
- TestComplete - A paid tool provided by Smartbear which can be used for automation of Web, Mobile as well Desktop applications.
- Telerik Test Studio - It is a paid tool that provides record and playback feature for automating web, APIs and windows applications.
- Watir - Watir (Web Application Testing In Ruby) is an open source automation tool that can automate web applications in Ruby.
- Appium - It can considered as Selenium for mobile applications. Just like Selenium, it is open source and has large user base.
- Robotium - It is also an open source tool using which we can native automate native and hybrid android applications.
- Ranorex - A paid tool with record and playback feature using which we can automate android, ios and Windows applications.
- Calabash - An open source tool by Xamarin using which we can automate andoird and iOS applications.
For more details and in-depth review of different test automation tools, check our posts on Test Automation tools for WebSites, Android Apps, iOS Apps and Web services.
Our Free Tutorials
Now, that you have fair understanding of the test automation process, you can select the right tool for learning automation. For this, I would highly recommend you to go through the post - Roadmap for transition from Manual to Automation QA. After that you can check our below mentioned tutorials on popular test automation tools.
I hope this article helped you in understanding the different aspects of automation testing along with the understanding of all the activities performed in the automation testing. Now, you can start your automation testing journey with our popular tutorials.