Test Techniques & Events Training

Table of Contents

Module Video Resources

Opening Remarks

Overview

Test Events

Understanding Test Events and their purpose

Unit Testing

Unit Test Demonstration

Integration Testing

System Testing

Regression Testing

Acceptance Testing

Test Execution

Test Flow

Continuous Testing

Test Methods

Black Box Testing

White Box Testing

Grey Box Testing

Smoke or Sanity Testing

Section 1

Module Video Resources

Opening Remarks

“Hello and welcome to this training on software testing techniques and events. We’ll cover the different categories of software testing, such as test events, test methods, and test techniques. I would like to start this session with some definitions.

• What is an event? A planned occasion. A thing that happens.
• What is a method? A particular form of procedure for accomplishing or approaching something. A technique.
• A way of carrying out a particular task. A skill or ability.

There are several different testing categories that can be used, depending on the goals of the testing, the type of software being tested, and the stage of the software development life cycle. Most of this training will address three of them. Yes, there can be confusion around the terms “test events,” “test methods,” and “test techniques” in software testing, and some people may use these terms interchangeably, but there are subtle differences. While these terms refer to different aspects of software testing, they are interrelated, and understanding the differences between them is crucial for designing a useful software testing strategy.

So, what are test events? Test events refer to the different stages or events in the software testing process, such as unit testing, integration testing, system testing, regression testing, and acceptance testing. They are typically executed in a specific order, with each event building upon the previous one to ensure that the software is tested thoroughly.

Well, what are the test methods? Test methods, on the other hand, are the approaches or strategies used to execute tests. They refer to how tests are designed and executed and include skills like white-box testing, black-box testing, and grey-box testing. You can even include smoke testing, and sanity testing.

Finally, what about test techniques? Test techniques are approaches used to design and execute software tests. Some of the commonly used test techniques include Equivalence Partitioning, boundary value analysis, Decision Table Testing, Exploratory testing, Risk-Based Testing, and more. Each technique has its benefits and can help uncover different types of defects.

It helps to break down these categories of testing into clear groups. When you are testing alone, you may not need to organize the testing concepts. You just need to have the skills working in your head. But testing is often a collaborative endeavor. Therefore, you need to have the skills clear and people thinking about test skills and how to use them in the same way. Otherwise, conflicts and confusion will hamper the test efforts.

Overview

Let’s continue with an overview of software testing. Software testing is the process of evaluating a software application or its component to identify any defects or errors. There are different types of software testing. There are different ways to approach testing. And there are different techniques that can be exercised to accomplish testing successfully. The single objective is to reveal defects and issues. It’s just that how you go about doing it is a flexible endeavor.

Test Events

So, let’s dive into test events. Test events are different stages in the software testing process, such as unit testing, integration testing, system testing, regression testing, and acceptance testing. Each test event has its purpose and aims to uncover different types of defects. Test events can be organized and executed in various ways, such as through manual testing or automated testing.

Understanding Test Events and their purpose

We defined these test events in the introductory part of this course. However, we shared the information in terms of types of testing. And I did not deep dive into each. Now, you should not be overwhelmed by each test event. And by the way what we are talking about here is a breakdown of functional testing, not non-functional testing.

Testing events exist with somewhat of an order of doing things. In other words, there is flow to the testing events to accomplish a software testing project.

Unit Testing

Unit testing is a software testing event used to test individual components or units of the software. This is typically performed by the developer, and it is focused on verifying that each unit of the software meets its specified requirements and functions correctly.

Unit testing is normally an integral part of the software development process, as it helps to catch bugs early and ensures that each unit of the software works correctly before it is integrated with other units. In the unit testing process, each unit of the software is tested in isolation, and test cases are created to verify that each unit meets its specified requirements. The testing activity normally happens in an IDE as the test environment. This is why mostly it is the developer that does unit testing. But as I will mention later, one of the trends in software testing is TDD. This is an acronym that describes defining test cases before writing code. It’s an interesting concept that can cause test analysts and developers to collaborate on program development and testing.

JUnit is a popular framework for unit testing in Java, and it provides a set of tools and annotations that make it easy to write and run unit tests. JUnit is easy to use and widely supported, making it a good choice for unit testing Java applications.

When developers use Microsoft languages, then there are two frameworks known as NUnit and XUnit. These support C#, C++ for .NET development environments.

For Python development there is UNITTEST and Pytest. Pytest provides much more informative feedback, which comes in handy when it’s time to debug. Both Pytest and UNITTEST are tools to use for automated testing in Python.

I will leave these automation frameworks for an advance course in test automation. Here I am only mentioning them.

Unit Test Demonstration

I have a test environment on my machine called PyCharm. It is one of the tools known as an IDE which is short for Interactive Development Environment. I like to call it my test environment because I not only develop code but often test code and applications. When you are testing code, you are doing unit testing. You may hear about another version of unit testing that is Component or string testing.

I am only going to give it a mention. Component testing is unit testing of one program, class, or function. While string testing is more than one program, class, or function together. You might hear the term stub where a developer creates a main program or called program as a stub to test programs calling or being called. Test analysts rarely do this.

So, I am going to take a few minutes to demonstrate unit testing. I will choose a couple of Python programs I wrote recently and show them to you. Then we can talk about the process of unit testing, so you get a more hands-on picture of how it works.

Integration Testing

Integration testing is a software testing event used to test how different components of the software interact with each other. This is typically performed after unit testing and is focused on verifying that the components of the software work together correctly.

Integration testing is an important part of the software development process, as it helps to catch bugs that are caused by the interactions between components. In the integration testing process, the components of the software are combined and tested together, and test cases are created to verify that the components work together correctly.

There are some interesting concepts associated with this brand of testing that is not always obvious. What I mean is that I have been on test projects that basically just used integration and systems as a way to go with two rounds of testing before turnover to the business for acceptance testing. But as mentioned the idea behind integration testing are component interactions.

This could be a foreground and background module where data is sent from the foreground to a background module to formulate and print a report. Or a process where data is passed from one application to another. For instance, a sales invoice application that passes financial data to an accounting application. This can be a module sending out notification data for printing, email, or web services.  I hope you are starting to get the idea.  This is why I like to consider interface testing a part of integration testing. Some will agree and some will not. And we can agree to disagree. The work still must get done. So, I am going to include interface testing with integration testing for this course.

I define interface testing as a process to evaluate whether systems or components pass data and control correctly to one another. Again, this is why I see the relationship of interface testing to integration. If I test two components to confirm they work together, it’s integration testing. If I test two components to find out if they pass data correctly to each other, it’s interface testing. Interface testing is a part of integration testing.

With that said, here is a list of flavors of integration testing. Keep in mind I started this argument mentioning that integration testing has some interesting concepts. Now I’m calling them flavors. So, either way these help describe implementation of integration testing. You will encounter some of them on a test project.

• Big Bang Method – This method involves integrating all the modules and components and testing them at once as a single unit.
• Top-Down Method – an incremental approach that involves joining two or more logically related modules for testing. The process normally involves using dummy programs called Stubs and Drivers to stimulate the behavior of lower-level modules that have been coded.
• Bottom-Up Method – tests the lowest components of a code base first. It involves taking integrated code units and testing them together, before testing an entire application.
• Hybrid Testing Method – takes advantage of both bottom-up and top-down integration. It is also known as sandwich testing.
• Incremental Approach – It is like the top-down method that involves joining two or more logically related modules for testing.
• Stubs and Drivers – it used in both top-down or incremental methods, where either a stub program or driver program is needed. It is usually stubs in top-down and drivers in bottom-up.
• Interface Method – I am referring to what some call interface testing that is important to integration within an application and between applications. It is used to ensure data communication is working for business needs. I mentioned earlier that it includes web services which means API testing is involved which has to do with several technologies like JSON, XML, SOAP, etc.

System Testing

System testing is a software testing event used to test the entire software system, including all of its components and their interactions. This is typically performed after integration testing and is focused on verifying that the software system meets its specified requirements and functions correctly.

System testing is an important part of the software development process, as it helps to catch bugs that are caused by the interactions between the software system and its environment. In the system testing process, the entire software system is tested, and test cases are created to verify that the software system meets its specified requirements.

Regression Testing

Regression testing is a software testing event that seeks to re-run functional and non-functional tests. The purpose is to certify that a software application works as intended after any code changes, system updates, environment revisions, improvements, or optimizations. The goal is to make certain that any change does not adversely cause functions that were working to quit working.

This testing event may be scheduled every time there is a new release of any software in the test or production environments. Because this can be a highly repetitive test event, this testing is a good candidate for test automation. It can significantly reduce the testing costs.

It is not uncommon that what a test team defines for system testing or integration testing is used for regression testing.

Acceptance Testing

Acceptance testing is a test event used to test the software from the customer’s perspective. This is typically performed by the customer or an end-user, and it is focused on verifying that the software meets the customer’s needs and expectations. When the test team are included they should change their approach of testing to find defects to verifying that the software application is meeting test scenarios based on business scenarios.

Acceptance testing is an important part of the software development process, as it helps to ensure that the software meets the customer’s needs and expectations. In the acceptance testing process, the software is tested from the customer’s perspective, and test cases are created to verify that the software meets the customer’s needs and expectations. Those needs and expectations might include checking statistics to confirm and add or update or delete worked. It might include validating an email was sent or a report was scheduled in the background. The test might need to confirm that specific data was sent to another application or that another application was used to get certain up-to-date data. This is not exhaustive. The business needs and expectations have a much broader scope than just checking the function is working.

Test Execution

So, test events can be organized and executed in different ways. Test events can be manual or automated testing, or a combination of both. Test events are also scheduled to meet developers needs and user needs. Sometimes a project elects to do all testing in the same test environment. This can create a challenge when developers need it, the test team needs it and the business users start testing before the test team is finished. This is typically identified as one of the risks. And the mitigation is to setup adequate test environments so that there is no traffic issues. But budget constraints can be an impact.

Test Flow

The flow of the Test Cycle is not an exact discipline. The order of things in testing is affected by several conditions. One condition is the programs as they are being developed. There may be some stand-alone programs or there may be some programs that run together in the development. Another condition is the programs once they exist. Do they go through some changes because of defect fixes required, or change requests or enhancement requests? Another condition has to do with the test environment. Are there configuration or OS-related upgrades? The course of test runs can change with these conditions taking place. Otherwise, the order of testing tends to flow with unit, integration, system, and acceptance testing. The only two testing methods that can iterate are the unit testing and the regression testing methods.

Continuous Testing

Nowadays you will hear more about the Agile way for continuous improvement, continuous testing along with continuous delivery. The key action is continuous testing (CT) where this is a software development process in which applications are tested continuously throughout the entire software development life cycle (SDLC). The goal of CT is to evaluate software quality across the SDLC, providing critical feedback earlier and enabling higher-quality and faster deliveries.

The method used is batching tests for processing overnight. Each team potentially can submit some automated test scripts to run during the night. On the following day developers, test team, and users can review their results. Thus, there is an alleviation of potential bottlenecks. This may not be ideal in every situation, but because Agile Scrum and Kanban dictate smaller scope of development and testing, this is possible.

Test Methods

Moving on to test methods, there are three main types of software testing methods: white box, black box, and grey box testing. However, I mentioned earlier that you can even include smoke testing, and sanity testing. These fit into the test method genre. White box testing involves testing the internal structure and logic of the software, while black box testing focuses on testing the software from the end-user perspective. Grey box testing is a combination of both. Each test method has its advantages and disadvantages, and it’s essential to choose the right test method based on the software’s requirements and goals.

The goal of Smoke testing is to verify the stability of an application. The goal of Sanity testing is to verify consistency. Software Developers or Test Analysts both can perform smoke testing, whereas test analysts alone perform sanity testing. Smoke testing is a subset of acceptance testing, while sanity testing is a subset of regression testing. This is one of those processes in testing that you will notice interchangeability. We don’t need to be anal; we just need to understand. For either approach I am asking the question, “Is the software application still behaving like we expect?”.

Black Box Testing

Black box testing is a testing method in which the tester has no knowledge of the internal workings of the software being tested. All that is known is what the business, functional, and test requirements communicate. The tester focuses on the inputs and outputs of the software, and test cases are created to verify that the software produces the correct outputs for a given set of inputs.

White Box Testing

White box testing is a testing method in which the tester has complete knowledge of the internal workings of the software being tested. The tester focuses on the internal components of the software, and test cases are created to verify that each component functions correctly.

Grey Box Testing

Grey box testing is a testing method that combines elements of both black box and white box testing. The tester has some knowledge of the internal workings of the software application but does not have complete knowledge. The tester focuses on the inputs and outputs of the software, as well as the internal components. It is not often that a test analyst has this privilege, but it can happen. As the TDD concept I mention takes hold of more projects, I believe we will see more of grey box testing.

Smoke or Sanity Testing

Smoke testing or sanity testing is the preliminary check of the software after a build and before a release. As it was mentioned, users may do sanity testing while test analyst do smoke testing when needed. If you have a concern that a build can end with error, then a smoke test can help verify the build is working. You are running just enough test cases to determine if the application is operational. Likewise, with sanity testing. If the users are ready to do another round of acceptance testing after some configuration changes were completed, they may begin with sanity test to confirm the environment is ready.

Closing

This is not the completion of this training module. There is a second part to this document. Please click here to see section 2 of this document. The video resource list is complete at the beginning of this document. Thanks for reading this training.