With a huge market share, Selenium is currently the uncontested king of the web-automation universe.
More than 25,000 software companies are using the Selenium framework. Since its first release in 2004, it has been continuously developing to support many languages, platforms, and browsers and assist the widespread use of the Agile methodology.
There are probably not too many professionals in the automated testing domain who haven’t used it one way or another. You can find 55K questions (!!) about Selenium in StackOverflow says a lot about its popularity.
As stated in this Microsoft blog, Selenium has clearly become the industry standard.
However, with great power comes great responsibility. And unfortunately, while being extremely popular and flexible, Selenium has some major drawbacks that can dramatically hurt its potential and reduce ROI.
Faults that are are too critical to be ignored??
While every test automation coder with some experience working with Selenium could easily make a list of its faults, not all drawbacks are equal. I will list ‘only’ three of them – but they are harrowing ones if you ask me and could lead to major losses to any company running test automation processes at scale.
- Flaky tests: Selenium’s possibly most critical fault is the relatively high percentage of flaky tests. Not being sure whether a certain result indicates a real problem is a cause for a lot of pain. No one is happy about a team playing a game of hide-and-seek with bugs that don’t actually exist (you can find common reasons for why flaky tests happen here). Flaky tests waste company time (waste its money), increase workload, delay releases, and build up a lot of frustration, which is never a healthy thing. Besides, since it’s been reported that in many cases, more than 20% of Selenium tests come up with false-positive results, testers tend to take the dangerous step of ignoring these results altogether. Which, of course, misses the whole point in QA testing…Agile became so popular because it’s designed to increase efficiency, support business KPIs, and grow profitability. Flaky tests that delay releases are hurting Agile and thus impacting your bottom line.
- Testing leads to the natural need to trace the root cause of an error, which completes the cycle. After all, one main goal in automated testing is to locate issues as fast as possible – and fix them as fast as possible. If testing is devoid of root cause analysis, it’s not as good as it can get. Again, it’s not a tiny fault. As with the other drawbacks, struggling to find the root cause could delay version releases and increase workload.
- Selenium tests that overcome the above usually do not run fast enough. In fact, you can say they are slow. In the stressful world of software, this is a major fault. Being limited due to slow test runs is not something you can ignore.
The ROI suffers – How can we work it out?
Can an organization work around these issues and somehow continue using Selenium effectively? There are a few ways it can, by not settling for these faults and finding ways to optimize tools and processes.
When dealing with measuring the ROI of automated testing to optimize it, Several ways come to mind. Here, for instance, you’ll find an in-depth analysis of six different approaches. Generally speaking, this resource suggests calculating the cost savings from reducing redundant tests, defect, and knowledge leakage and from ensuring proper test coverage while weighing it all against the investment (hours needed for development, execute and maintenance of each test) that go into these practices. “The ROI in test automation can be huge,” states the article, and rightly so. Accordingly, companies should aspire to minimize the significant negative impact of Selenium faults on its ROI.
Ways to optimize your Selenium automation ROI
Do nothing and hope for the best.
So, what do companies usually do? Not all companies were created equal, and different companies have different strategies for dealing with these drawbacks. Some companies do nothing. They’re probably aware that Selenium is a mixed bag, but they choose to bury their heads in the sand. They probably have their own reasons.
Add your own code
Other companies take a more proactive approach and struggle to develop code improvements for the Selenium platform independently. Why did I write a struggle? This is a costly and complex mission that can become a real burden on a company’s budget and workload (consider that ongoing maintenance is needed).
Leave Selenium behind
A third approach is to use Selenium-based external test automation tools that are mostly designed for “codeless experience,” thereby forcing users to leave Selenium coding behind and move to their realm.
These tools are abundant, but preventing coders from using Selenium like they are used to is a disadvantage. Besides, these tools can’t support existing (legacy) tests by their very nature.
Add a Selenium optimization layer & keep coding as before
A new and emerging approach is to add a transparent optimization layer on top of Selenium and keep working. That’s the approach we follow here at RadView. We believe it offers the best alternative for Selenium coders. This approach brings quite a few benefits when compared with the previous approaches. First, it eliminates flaky tests (we use AI for that). Second, it speeds up test execution. Third, it automatically creates root cause analysis reports, thereby reducing the need for a redundant search. And finally, it optimizes legacy tests instead of external solutions, which forces users to recreate them if they want them to be optimized.
To summarize
Selenium poses a real opportunity, but it includes challenges that impact companies’ bottom line. Since there are clear ways to optimize it, and the impact on the ROI should be thunderous and clear, I think it’s a no-brainer.
