Test Management Training 1

Table of Contents

Opening Remarks

Introduction to Test Management

Test Management in Agile versus SDLC

Test Management Objectives

Understanding the Process of Test Management

Benefits

Challenges

Test Management Tools

Risk Assessment

Test Management Tool Categories

JIRA Test Management

ALM Test Management

Selecting Testing Tools

Evaluation

Module Video Resources

Section 1

Opening Remarks

The objective of this training module is to train you in the art of managing testing resources. In test management, usually a test manager or team lead can closely organize, oversee, analyze, and track the testing activities. But the tool is made available to all project members. So, it is necessary that you understand how test management works.

If you’re not yet familiar with the role test management plays on a test project, this training is here to help.

By the end of this training, every student should have a much better understanding of the automated and manual test management process. It is my expectation that you are familiarized with the processes to contribute on a test project with some level of skill. The primary benefit of this training is for test analysts who need to know how to handle test activities on a software test project.

This module includes lectures, a lab session, and a homework assignment. It is expected you will find benefit from all methods that we use to disseminate training information. I feel the need to remind you how we will provide you with information on the front-end. But we won’t stop at that. At the back end when you have completed this training, you will be provided with access to training videos that will continue to support you with ever-green knowledge about the topics of this course.

Introduction to Test Management

Test Management is an oversight process that involves the coordination of all testing activities, including test planning, design, execution, bug tracking, test case development, status reporting, and closure. Effective test management helps to ensure that testing is monitored and completed efficiently and effectively, and that the final product meets the quality standards expected by stakeholders.

Test management can be implemented manually or with an automation tool. When a development project is small and testing requirements are simple, doing this process manually is adequate. But if there is complexity, such as resources in multiple locations and test requirements not simple, test management should employ the use of an automation tool.

The manual approach to test management can usually be handled with some manual tools like MS Word, Excel, and PowerPoint. The test plan can easily be written using Word. The test case development and bug management can easily be managed with Excel. And then any reporting can be shared using a tool like PowerPoint slide presentation.

But test management software has come to the forefront. Two popular tools that is widely used in the market are JIRA and ALM. Large companies gravitate to ALM while smaller companies engage JIRA. Both tools have their version of test management features and functionality. A test plan can be entered and kept evergreen. Test cases can be entered and scheduled for execution processing. As test cases are executed it provides support for capturing status data to pass or fail test steps. If defects are encountered, it allows defect entry and follow-up tracking until the defect is closed or deferred. The test management functions allow for reporting support. Of course, it keeps a history of what happened, and some data can be reused for another round of testing.

Test Management in Agile versus SDLC

Test Management should be considered the same in all SDLC models. However, what is different is scope and duration. Normally with SDLC the test phase is end-to-end. Once testing begins it is continued until all test events are completed. However, with Agile the scope and duration changes from iteration to the next. The iteration is a 2-to-4-week span to cover a function or component of an application. So, the scope is much smaller. Therefore, the scope of test cases is much reduced and thus the time it takes to complete is shorter.

With Agile it is very likely a test scenario is the scope of the testing. So, the number of test cases is low. The amount of time it takes to complete is low. Reporting and bug tracking should be significantly lower in comparison to SDLC.

Test management can be approached differently in Agile and other SDLC methodologies. In Agile, testing is an integral part of the development process and occurs continuously throughout the development lifecycle. In SDLC, testing is typically performed after the development phase is complete and before the product is released to the market.

In both Agile and SDLC, test management involves the coordination of all testing activities, including test planning, design, execution, and closure. The key difference between the two approaches is the focus and timing of testing activities. With SDLC there may be four iterative phases of testing back-to-back. Integration, system, regression, and acceptance testing phases are likely. Meanwhile Agile reduces the testing to regression testing as part of a CI/CD automated process unless manual testing is requested.

Test Management Objectives

Test management is the process of planning, coordinating, and controlling testing activities. Test management involves the use of various tools to ensure that testing is conducted efficiently and effectively. In this training module, we will explore the key concepts of test management and the tools used in the process.

The Learning Objectives are:

• Understand test management importance
• Learn the key processes involved in test management
• Understand the role of different types of testing tools in test management
• Learn how to select appropriate testing tools for your organization

Understanding the Process of Test Management

Test management involves defining the testing objectives, identifying the scope of testing, defining test plans and strategies, selecting appropriate testing tools, managing the testing resources, tracking, and reporting testing progress, and ensuring that defects are identified and resolved.

Manual application – Test Management often is the oversight of test plan entry, test scheduling, bug reporting, the logging of test results, test tracking, incident management and test status reporting. It typically includes the process of drafting test plan components, and development of test cases.

Automation application – automated test management is accomplished using a software tool for all the processes mentioned for the manual application. The great benefit of using a tool is intranet or internet access to all test management data. Popular tools in this space are ALM and JIRA. But there are many others, both open source and commercially licensed.

The general goal of test management is to allow teams to plan, develop, execute, and assess all testing activities within the overall application development effort.

Benefits

Following Test Management practices is very beneficial to the development project and the test project. Effective test management can provide several benefits to the testing process, which include:

1. Improved software quality: The use of Test management ensures that software is thoroughly tested. This can help identify and resolve defects before the software is released. This also helps improve the quality of the software product and reduces the likelihood of defects and issues being discovered after release.
2. Reduced development costs: I hope this point is not boring you. Early detection of defects in the testing process helps reduce the cost of fixing them later in the development process or after release. This can result in significant cost savings for the organization.
3. Improved testing efficiency: Test management practices provide a structured approach to testing, which helps ensure that testing activities are conducted professionally and successfully. This can help reduce the time and effort required to complete testing activities.
4. Enhanced collaboration and communication: Test management involves coordination between various stakeholders, including developers, testers, project managers, and other team members. Effective test management encourages collaboration and communication among team members, which helps ensure that testing activities are aligned with project objectives and goals. This a great benefit of using a tool that gives you and your team intranet or internet access to all test management data.
5. Better risk management: Test management helps identify and manage risks associated with software development and testing. This can help reduce the likelihood of critical defects being missed and can help ensure that potential risks are identified and addressed before release.

Risk assessment or evaluation is the process of three actions: risk identification, risk analysis, and risk mitigation. When a test team is doing planning, this is one element that should be included in the test plan. Ask the question, “What are potential risks to successful testing?”. Testing can pose risks to the development project, the test project, and test resources. Consider the following grouping of possible risks, and then think about questions to ask to identify potential risks:

1. Financial Risks: Financial risks involve a project’s monetary factors.
2. Strategic Risks: Strategic risks involve the strategies chosen to complete a project.
3. Performance Risks: Performance risks involve the overall project and application performance.
4. Security Risks: Security risks involve the project’s vulnerabilities.

Challenges

Test management involves several challenges that can impact the quality and effectiveness of testing. Some of the key challenges in test management include:

1. Resource management: Managing testing resources can be challenging, particularly in large projects where there are multiple teams and stakeholders involved. The scope of resources includes people, time, and assets. Ensuring that testing resources are effectively utilized and properly allocated can help improve testing efficiency and effectiveness. But challenges arise when people are not available, and testing needs to proceed. Time challenges can arise when the application test environment is down at the time testing needs to be proceeding. And then there are other challenges that can arise if there is a problem with data. The data requirements have changed and there is some delay with synchronizing the data for the scheduled testing.
2. Test environment management: Managing test environments can be challenging too. There are times particularly in complex systems where multiple environments are required for testing. Ensuring that test environments are properly configured and maintained can help ensure that testing activities are conducted in a reliable and consistent manner. But what if there is a breakdown in one of the environments. Are safeguards in place to quickly recover? This is one of the reasons risk assessments have importance.
3. Test data management: Managing test data can be challenging, particularly in systems that generate large volumes of data. Ensuring that test data is properly managed and maintained can help ensure that testing activities are conducted with realistic and representative data. Data integrity issues is not good for the test project. Proper version control and data backups are important to minimize data integrity issues.
4. Test automation: Test automation can be challenging, particularly in complex systems where there are multiple interfaces and integrations. Ensuring that test automation is properly designed, implemented, and maintained can help improve testing efficiency and effectiveness. Because of the rapid changes trending with test automation frameworks, The tools selected must be able to keep up. I am adamant about selecting test tools that are designed with interfaces to other tools. This allows for being able to use alternative methods when a tool is not directly meeting your test needs.
5. Defect management: Managing defects can be challenging, particularly in large projects where there are multiple defects being reported. Ensuring that defects are properly tracked, prioritized, and resolved can help ensure that software applications meet the highest standards of quality. This is why severity and priority status are helpful.
6. Test reporting: Test reporting can be challenging, particularly in large projects where there are multiple stakeholders involved. Ensuring that test reports are properly generated and distributed can help ensure that stakeholders are informed about testing progress and results. With large projects it is expected that frequent meetings are scheduled to ensure stakeholders are kept informed.

Test Management Tools

Test management tools are not the same as test case management tools. The latter is focused on automating the process of developing test cases with steps and executing them. While that is good, it is not the scope of test management. Test management tools have a broader scope which is to support all the test plan components including entry, update, coverage analysis, and reporting.

One manual test management process that is still not supported well in a tool is risk assessment. Before I jump into talking about risk assessment, I want to mention this. There are three types of risks to assess from a test project perspective:

1. Project risks have a negative impact on project outcomes. A test project can have its own risks just as the development project. So, it must be reviewed for potential showstoppers.
2. Product risks threaten application performance and increase the possibility of not solving a user problem. Mitigation efforts usually require proper system settings and configuration along with performance testing efforts.
3. Business risks influence the financial value of an application and affect the company’s revenue. Integration and system testing must have the business user in mind even though functionality is the primary challenge. What do I mean regarding the business user. If a user starts using the application for conducting the company’s business, there is money involved. Everything must work within the user’s workflow. Otherwise, money and customer could be lost.

Risk Assessment

Risk assessments are done on many projects. It is a valuable tool to have as a project is in its inception. This is no exception for a test project. You have several tools that can be critical to the success of the test project. Let me list a few: your test data, the database itself, the test environment, your test documents (plan, test scenarios, test cases, etc.), the team’s health and availability. It is imperative to do a risk assessment to have the team understand the potential risks your team faces with the project.

The risk assessment document is not only intended to identify the potential risks, but to communicate what is in place or will be in place to mitigate or reduce the impact if the risk happens. The objective of mitigation is to lessen the impact or remove the impact.

Looking at the column headings, let’s discuss meanings to ensure you understand what information is appropriate for each column. The risk ID can be a number or formatted alphanumeric identification. Of course, the risk description should clearly indicate the potential risk. Date of entry is the date this item was added to the document. Priority and Severity should have values Low, Medium, or High. For the priority column this indicates how important the risk is to address. The severity column indicates how much damage the risk can cause if not handled properly. Once the risk is analyzed taking the priority and severity into account, a proper mitigation can be specified. This can be a reference to some process already in place to minimize or eliminate the risk. Otherwise, this cell can be used to specify steps to take for mitigation. The status cell should be used to identify if follow-up is required. Enter “Good” if no follow-up is needed. Enter the assigned person’s name that will be assigned to resolve.

Test Management Tool Categories

It is necessary to divide the test management tools into three categories. If you are new to the automated test software, it can be confusing to hear about bug trackers, test management, and test case management tools. Be aware that there is often a connection between these tools. But it is helpful to understand the space they fit in, and which ones do integrate with each other.

Jira is the tool that integrates with so many other test management, case management, or bug tracking tools. This is what makes it in high demand. Whereas ALM is a powerful test management tool, it has not reached as many project sites as Jira. And ALM and Jira can integrate.

I have listed here some tools that fit into the three categories. As you work on various test projects it is very likely that you will encounter these tools. You may even be asked to participate in tool evaluation activities to decide on tool selection. The course will expose you to ALM, Jira, QA Touch, and Zoho Bug Tracker. Having that knowledge will provide you with familiarity enough to be helpful on such a task.

In the future when you hear about test management tools know that Test case management tools are that which help to manage and organize the test components like test cases, test plans, and test execution. Test case management tools help software testers to manage test cases and reduce the manual efforts to ensure thorough test coverage. On some projects there is only a need to manage defects. That is where investment in a bug tracking tool is sufficient.

JIRA Test Management

JIRA provides support for agile development methodologies such as Scrum and Kanban, which are commonly used in software testing. This enables teams to work more efficiently and effectively, by providing a flexible and adaptable framework for managing the testing process.

JIRA is also a popular issue tracking and project management tool developed by Atlassian. While JIRA is primarily designed for managing software development projects, it also offers features and functions to support test management. Some of the key features and functions in JIRA that support test management are:

1. Test Planning: JIRA provides the ability to plan and organize tests using customized workflows, which can be designed to suit the needs of specific testing processes.
2. Test Execution: JIRA offers a flexible and customizable platform for managing test execution. This includes the ability to create test cases, track test execution progress, and report on test results.
3. Defect Management: JIRA provides a robust defect management system, allowing testers to report, track, and manage defects. Defects can be linked to test cases or requirements, prioritized, assigned, and tracked to closure.
4. Traceability: JIRA provides a comprehensive traceability feature that allows testers to track the relationship between requirements, test cases, and defects. This ensures that all requirements are tested, and defects are fixed before the application is released.
5. Reporting and Metrics: JIRA provides a variety of reporting and metrics options to track testing progress, test coverage, defect trends, and other relevant information. Custom reports can be created, and dashboards can be configured to display key metrics.
6. Integration: JIRA supports integration with a variety of other tools and systems used in software development, such as source code management, build tools, and automation tools. This integration helps to create a seamless software development process and reduce manual effort.

I have only had experience using JIRA for defect management. I used its capability for about a year only on one test project. It did not take long to grasp its features.

ALM Test Management

ALM provides various features and functions to support test management, including test planning, test execution, defect management, traceability, reporting, metrics, and integration. This software is developed by Mercury Interactive but marketed now by OpenText who recently bought Micro Focus.

Application Lifecycle Management (ALM) is a comprehensive set of processes, tools, and methods for managing the entire lifecycle of a software application, from requirements gathering to development, testing, deployment, and maintenance. Test management is an essential component of ALM, and ALM tools provide various features and functions to support test management. Some of these features and functions are:

1. Test Planning: ALM supports test planning by providing features to define and manage test plans, test cases, and test suites. Test plans can include the scope of testing, test objectives, and timelines. Test cases can be created, reviewed, and managed, and can be linked to requirements, defects, or other artifacts in the application.
2. Test Execution: ALM supports test execution by providing features to execute test cases and record the results. Test execution can be automated or manual, and ALM tools can provide support for both types of testing. ALM tools also allow testers to collaborate and share test execution results with other team members.
3. Defect Management: ALM supports defect management by providing features to report, track, and manage defects. Testers can report defects and link them to the corresponding test case or requirement. Defects can be prioritized, assigned, and tracked to closure, and can also be linked to other defects or requirements.
4. Traceability: ALM supports traceability by providing features to trace test cases to requirements, defects, and other artifacts in the application. This helps in ensuring that all requirements have been tested, and defects have been fixed before releasing the application.
5. Reporting and Metrics: ALM supports reporting and metrics by providing features to generate reports and metrics on test progress, test coverage, defect trends, and other relevant information. This helps in making data-driven decisions and improving the overall quality of the application.
6. Integration: ALM supports integration by providing features to integrate with other tools and systems used in the software development process, such as source code management, build tools, and automation tools. This helps in creating a seamless software development process and reducing manual effort.

ALM is a tool that has dominated the test market for years. It can be an expensive investment, but a tool that meets the challenge from project to project. I have worked with this tool on many projects since year 2000. I have been called upon several times to upgrade companies to new versions of the product. Some have waited until the upgrade was outside an easy migration path due to old technology issues. But I was able to upgrade the installation successfully but sometimes it took multiple days and nights to overcome the obstacles. The key is to never give up.

Selecting Testing Tools

Evaluation

Software evaluation is another form of testing. Try not to shy away from this kind of opportunity. It is the process of measuring the performance and utility of software. The purpose of software evaluation is to understand the value of the software being tested. Without going into a full lesson on the subject let me expound.

Let me give you some measurements to use in evaluating software. But first you need to identify what you are looking for. What are some requirements you need fulfilled? Then here are the features to measure:

1. Ease of use – How easy to use is the software? Can you make it work at first try?
2. Ease of learning – How intuitive is it? Do you constantly have to consult documentation?
3. Ease of implementing – How much time is necessary for setup? Is it hard?
4. Capability of functions – How much functionality is included? Is it powerful?
5. Flexibility of interfaces/integrations – Is it a software island without bridges to others?

Now take those measures and assign a value of 1-10 for the software under test (SUT). Then decide what percentage is acceptable for you if you are just measuring one tool. If you are measuring multiple tools decide on what percentage is acceptable to designate a passing score. Be difficult. Don’t be too difficult. Maybe you should set the mark at 90% instead of 70%.

Software evaluation testing does not have to be too hard and too long. But if the cost is significant, then you want to get some formal training with more in-depth measuring tools.

This is not the end of this training module. Please click here to access section 2 of this module.