Overview

Once you have decided to automate your testing activities, what then?

We are finally in a season where many companies are thinking about automation testing and testing frameworks. Test Automation tools have been around for several decades. Commercially licensed and open-source tools have been available and into significant number of releases. For example, Micro Focus UFT is an application functional test automation tool that is in its 16th release as of early 2022. This is not an indictment on the tool, I think it has served the IT community well. I certainly have made use of the tool to encourage application management personnel to endorse the tool for test automation.

Test tools designed for automation are brought in on major application development projects, but they don’t seem to remain engaged. The application maintenance teams drop it and go back to manual test efforts. A study concluded that time lost on repetitive tasks that could be automated is over 40 percent. Testing is a repetitive task that benefits from automation. A quote from Quali.com is “One of the most common issues is that the test part of test automation is rather straightforward to automate, especially with today’s abundance of great automation tools. But jumping straight into automated testing leaves you with a big heap of automation scripts, and often not the best ROI. Understanding why requires looking into the automation process.”

Let’s pause and think about that. How important is the automation process? I worked for a major oil company for over 20 years. One of my responsibilities for just under 8 years was running a testing quality assurance service. The service was global. The team started with 2 people and grew to 8 people distributed across the globe. The reason the team was formed came about that every time a new major application development project began, there was time and money spent on a feasibility study for test tools, purchase of licenses, product experts brought in, and when the project was over the tools were retired. I convinced my management we could do better. I launched a campaign to lasso the licenses as we started a test service. Some of them were still under maintenance agreements. We worked with the product vendors to combine and improve maintenance agreements. We also wrote articles including tips on product use. Then we leased out the tools based on project needs. The savings was calculated at around $100K annually.

Though the effort was successful, there was still a problem. The testing tool licenses came back into our hand usually when the project was over. Maintenance teams did not have the budget to keep the tools in use. The test scripts often went to waste.

Maybe the test service needed expansion. Test script development in the hands of a project is limited to the scope of the application needs. A test service script development team might find synergy between projects and develop scripts that are more generic. Project test analysts could be used to define test requirements (define test cases) that the test service team would automate. Now the test automation scripts are the property of the test service to provide ongoing script maintenance. Now the test service can provide ongoing CI/CD pipeline testing services.

CI/CD is defined as a method to frequently deliver applications to customers by introducing automation into the stages or releases of application development. More specifically, CI/CD introduces ongoing automation and continuous monitoring throughout the application lifecycle, from integration and testing phases to delivery and deployment cycles. The test service team can become responsible during application maintenance cycles and keep reusing the test script investment.

I am sure that others have entertained different possible solutions. But what we tolerate in our major application development projects, is not usually tolerated in the application maintenance and support environments. Budget seems to be the deciding factor. Test scripts need a home where they can be nurtured. That word includes the word developed. Test scripts need to be developed into artifacts that are reused on multiple projects, where a large percent of the code is reusable across projects and applications. Return on investment (ROI) is key to the budget issue.

Decision Points

Test automation is supposed to be a directive that results in a cost-saving endeavor. For cost-saving activity to happen, there is more than one decision to consider that finalizes the course of actions. Let’s consider the following:

1. If the testing activities can happen in multiple geographic locations, leverage the time zone differences pertaining to personnel compensation costs, and pertaining to test execution availability times.
2. While the team’s work oversight can be centralized, the team can be decentralized.
3. Maybe a combination of tools and programming languages will be needed to meet the scripting demand. There are languages and test tools that interface with various libraries and API’s that provide support for interfacing with the technologies in use by many applications.
4. What item 3 leads to is the understanding “one size fits all” thinking is not a good answer to address choosing the right test tool or programming language. I have managed well with using UFT, Excel, and LoadRunner for commercially licensed test tools. But then for open-source tools, I have used Python, Robot Framework, Visual Basic, and JMeter. The combination is not bad to meet most testing challenges. But I know other test engineers have selected a different combination of test tools to meet the project needs.
5. BlazeMeter, AUTIFY, Work Soft, and Test Project are a relatively new set of test tool solutions. They are among test solutions that tout “code less” testing. Maybe this brand of test solutions can support the test center concept better in the near future.

I think I was on the right track with the idea of having a central test center of expertise. But some other concepts need to go along with it to achieve higher levels of success.  For example, the COE team should be knowledgeable in testing concepts, but not necessarily strong programmers. That is where using the “code less” test scripting tools might be a good combination for long-term success. In any case, the testing world is going through changes due to test automation demands, AI, Machine Learning, and new cloud technologies. Let’s hope we make the right decisions for automation.