Test Automation has been a part of the Dev/QA cycles since the 90s. In recent years, the demand for faster deliveries without compromising quality has made automated testing a basic requirement in every software project, making it more and more dominant in the testing activity.
In the past, the end to end tests were conducted only towards the end of a release at the stabilization phase, after most of the development was done.
With the adoption of agile methodologies, the software is kept stable throughout the development phase and testing has become part of the development phase, without introducing long stabilization cycles at the end. The introduction of continuous integration and continuous delivery has created the need to continuously test the software to validate that it is fully functional after each change so it can safely be pushed to production. All of this cannot be achieved without most of the testing activity being done automatically.
With the growing needs for automation, new practices and technologies are emerging. In this blog we will describe some of the latest trends, needs and characteristics of modern automation testing tools.
Automation as part of Continuous Integration (CI) and Continuous Delivery (CD) Pipeline
Once having proper automation tests, they play a significant role in the development cycles and dev pipeline and are executed after the unit tests. There is a major difference between automation tests and unit tests. While in unit tests, only pieces of code are tested, for automation test a ready working environment is required. This requires building and deploying the application as part of the pipeline, prior the execution of automation tests.
Behaviour Driven Development (BDD) and automation
One of the major trends today is that tests’ definitions are created by the business owners and product managers even before the product is ready. In the past, the QA was responsible for defining the tests and building the scripts. BDD tools like Cucumber and JBehave are taking the lead in this market. BDD tools give the non-technical user the ability to define user scenarios in structured English. Later on, the developers take these definitions and develop proper tests accordingly.
For example, Cucumber’s BDD scenarios are written with Gherkin (Gherkin is the structured language of Cucumber). The developers use the descriptive language to write a working automation code, mainly using Selenium. Selenium is an open source and is the most common browser automation driver. It is widely used and is becoming the de facto standard for automation.
Browser Level Testing
As automation is replacing more and more the traditional manual QA, there is a need for it to test applications in a way that is closer to how the real user will experience them, which means using a real browser as part of the testing.
As mentioned earlier, Selenium is the most common browser automation driver these days. Many different frameworks and solutions exist on top of Selenium to simplify working with it – webdriver.io, nightwatch.js, protractor, nightmare.js, etc.
One alternative to Selenium which is gaining traction is Puppeteer – a new solution from google, focused on driving a headless chrome.
Cross Browser/Device testing
One of the time-consuming QA tasks is testing applications on more than one browser or device. Setting an endless matrix of combination of browser, OS and device is both expensive and time consuming.
Solution like saucelabs, browserstack, crossbrowsertesting and more, allow testing end to end scenarios using many browsers, machines, and devices in parallel. These tools supply the testing infrastructure.
Codeless
Writing test scripts isn’t an easy task for trained teams and is almost impossible for an untrained person. Expert abilities are required to create Selenium scripts. Companies like TestCraft, mabl, and Testim.io are working on technologies for codeless testing. Some of these vendors are using Selenium as the underlying execution engine, but the user is not required to understand Selenium or even know it is there. For the user – it is codeless.
Using AI to reduce maintenance work
One of the difficulties of test automation is the high cost of maintaining working scripts. This is especially true when doing E2E testing in browser level.
Every change to the code is potentially breaking the automation and the maintenance of the automation scripts to keep them
Today, new vendors in the automation area are adding AI that identifies changes automatically and aims to make scenarios much more stable than before. For example, Applitools focuses visually on the distinct changes by AI – in order to save the “noise” of many irrelevant changes. Testcraft take the AI to a different direction and identifies the selenium locators automatically – this helps tests’ flow become much more stable and immune to changes.
Visual Comparison
Visual comparison as an automation approach, it is not a new idea. However naive implementation of visual comparison are very fragile as every minor change in the UI breaks the test, so this approach was mostly abandoned.
Modern development in Machine Learning techniques have introduced new ways to solve this problem which brings Visual comparison as a viable option back to the table. One Vendor offering such a solution is Applitools.
Testing Performance as part of the automation suite
Performance testing or load testing is a way to test applications when it is experiencing load. Traditionally, the tools work on the protocol level, as this is the most effective way to load servers. Adding automation testing tools allows to see how the application is really working under load, from the end user perspective, measure the users experience, and not only learn how the APIs’ response is behaving. This is why tools like WebLOAD support automation testing tools such as Selenium.
To summarize, although automation testing tools are not new at all, the new trends change the existing tools and introduce new ones and new technologies that try to meet the new DevOps and shift-left challenges.
