Table of Contents
Introduction
Playwright and Selenium both serve the same purpose for web test automation. They provide an application programming interface (API) to test web objects. But they approach it differently. Playwright is a new and modern automation tool that provides a high-level API for testing web applications which makes it easier to write and run tests. In contrast, Selenium is a well-established and widely used browser automation tool that offers a low-level API for automating web applications, making it more flexible and customizable. But it also requires more time and effort to set up and maintain tests. Together, Playwright and Selenium provide a comprehensive and powerful toolset for web test automation, covering a wide range of use cases and requirements.
When a tool covers a wide range of use cases and requirements, it offers a more comprehensive and versatile solution. It meets the needs of a diverse set of users and applications. These tools have a flexible and robust architecture, as well as a variety of features and functionalities that allow it to adapt to different testing scenarios.
Test Benefits Analysis
The acceptance (ACPT) and end-to-end (E2E) testing of Salesforce applications is an important aspect of the software development process. I pick on Salesforce because it is a web application that is complex and challenging to test engineers. ACPT and E2E testing ensure that the application meets the customer’s requirements and is functioning correctly. Both test phases are also highly repetitive. With the increasing complexity of Salesforce applications, manual testing is becoming increasingly difficult, time-consuming, and prone to human error. This is where automated testing tools like Playwright and Selenium can step in, providing significant benefits for the ACPT and E2E testing of Salesforce applications.
Testing an application like Salesforce with tools like Selenium and Playwright can enable a test project to possess the following attributes:
-
- Improved Accuracy and Reliability
- This is one of the biggest benefits of using automated testing tools like Playwright and Selenium.
- Automated tests can be run multiple times, ensuring that the application functions as expected and reducing the risk of human error.
- This results in a higher degree of confidence in the application, as well as improved reliability and stability.
- Increased Efficiency
- Manual testing can be time-consuming, especially when there are numerous test cases to be performed.
- Automated testing tools like Playwright and Selenium can significantly reduce the time and effort required for acceptance and E2E testing.
- They can run multiple tests in parallel, providing fast and efficient results. This saves time and resources, allowing teams to focus on other important tasks.
- Improved Test Coverage
- Manual testing is limited by the resources available, such as, time, people, and equipment.
- Automated testing tools like Playwright and Selenium can cover a much larger number of test cases and scenarios, providing more comprehensive test coverage.
- This helps to identify potential issues early in the development process, reducing the risk of costly bugs in production.
- It allows for more quality time to devote to bug status reviews.
- Reusability
- Another key benefit of using Playwright and Selenium is the ability to reuse test cases.
- This can be especially beneficial for Salesforce applications, which often have complex workflows and business processes.
- Automated tests can be used repeatedly, saving time and effort in the testing process.
- Enhanced Test Maintenance
- Salesforce applications and others are constantly evolving and changing.
- Playwright and Selenium provide a flexible and maintainable testing framework that can adapt to changes in the application.
- This makes it easier to keep tests up-to-date and maintain the quality of the application over time.
- Improved Accuracy and Reliability
The use of automated testing tools like Playwright and Selenium provides significant benefits for ACPT and E2E testing of Salesforce applications. These tools help ensure that applications meet customer requirements and function as expected. As such, they are an essential part of any modern software development test process.
Browser Coverage
After using Selenium for a few years, it started to become a challenge making sure that the emulation drivers used by Selenium were up to date. When they were not you would encounter errors. That was not the kind of error appreciated when you are on a schedule. Once Playwright came on the scene this issue was addressed by the new tool on the block. It may have prompted a change in Selenium. Playwright and Selenium both handle browser interface in different ways.
Playwright uses a high-level API which abstracts away the complexity of the browser interface. This makes it easier to automate web applications and perform tests, as users do not need to have a deep understanding of the underlying browser technology. Playwright also provides built-in support for multiple browsers, including Chrome, Firefox, and Safari, and it can automate web applications across multiple platforms and devices. It also makes it easier to handle the need in test to open multiple tabs in a browser and test multiple pages.
Selenium, on the other hand, provides a low-level API for automating the browser interface. This means that users have more control over the browser, but also requires a deeper understanding of the underlying technology. Selenium supports multiple browsers, including Chrome, Firefox, and Internet Explorer, but it requires additional components to be installed for each browser. Selenium has now added a new Web Driver manager called Webdriver-Manager. There is a library and class to import along with an install statement you add to your code. It supports multiple languages, and it provides support for keeping emulation browser support current.
So, Playwright and Selenium provide flexible API’s that give users control over the browser interface. Both tools have their strengths and limitations, and the choice between the two will depend on the specific requirements of each test project.
Language Support
Playwright makes it easier to write test scripts, and it can generate test scripts in Node.JS, ,NET, Java, Pytest, and Python. Playwright’s API provides a clear and intuitive way of automating web applications, making it a great choice for those who are new to automated testing. Its code generating ability is powerful. The Node.JS support includes Typescript and JavaScript. But I especially like its support for Python and Pytest.
Selenium, on the other hand, supports multiple programming languages too. Its support coverage includes Java, C++, Python, and Ruby. Selenium is one of the most widely used open-source web application testing tools. If you have been involved in automation testing, you should be familiar with it. Some folks on the internet are thinking that it won’t be going away anytime soon. But Playwright is gaining ground. I still think there is room for both. Especially if your test environment already has existing selenium test scripts in use and working.
Selenium Noteworthy Plugin
Yes, there is a Selenium plugin that helps detect locators for Salesforce applications. The plugin is called “Selenium IDE” and it provides a set of utilities that make it easier to automate Salesforce applications using Selenium. The plugin is a Chrome and Firefox extension. There are also plugins for other web applications too. The “Selenium IDE” plugin helps users to easily find and identify locators for elements within a Salesforce application, making it easier to write automated tests.
The Selenium IDE plugin is designed to work with the Salesforce Lightning user interface and provides a set of helper methods that simplify the process of finding elements in a Salesforce application. This plugin is especially useful for those who are new to automated testing, as it helps to reduce the learning curve and makes it easier to get started with automated testing of Salesforce applications.
To learn more about and use the Selenium IDE plugin to develop Python scripts, you go to the following website:
- On the top right select PLUGIN for getting started
I’m including a sample script that I recorded using the Selenium IDE. I will also display a screen print of the tool and help you become familiar with it.
Selenium IDE Extension Screen Print
The sample screen of the Selenium IDE above has some obvious features. You can assign and list multiple test scripts to a test project. When you select a test script on the left side list, then you will see what the target URL is for the script. You will also see the first few lines of the script in the Selenium IDE format. You can scroll through the script code. If you desire you can create new projects and scripts, open and close existing projects, and save projects.
Hovering over the various symbols on the screen allows you to see more functionality.
-
-
- The arrows support executing one or more test scripts.
- The REC button allows for recording new scripts.
- The symbols in the top right corner support project controls
- The left side of the IDE supports test script management
-
One of the options from a right-click on each test script listed is a selection menu to export the test script. The popup window shown below is an example. So, from this option you are able to convert the Selenium IDE script into other languages.
This is a time saver. And it keeps Selenium competing with the functionality of Playwright. However, I must warn you here. It is great to generate the code in another language, But it is expected that the converted code will function without errors.
I generated my sample Salesforce script using the Python Pytest option. I was happy that the code came with comments and all the library references. But the execution failed.
I was able to get the script to work with some modifications. But the logout functions are still not working. Let me finish documenting more about the Selenium IDE capabilities. And then I will share my findings with the converted code.
There is a website that shares everything you want to know about the Selenium IDE. The setup details and execution options are there. The URL is https://www.selenium.dev/selenium-ide/
I can say that I like the fact that the code generated in the IDE to test Login and Logout of Salesforce works fine repeatedly.
I then discovered that there is another version of Selenium IDE which runs on the command line in my test environment (PyCharm). There is some setup required which is straight forward. This version of the IDE is known as the Selenium Side Runner.
Here is the setup requirements that must be executed on the command line in your test environment:
-
-
- npm install node
- npm install -g selenium-side-runner
- If you don’t already have a webdriver like chrome: npm install -g chromedriver
-
I caution you to be careful. If you installed some webdrivers on your machine previously and you added a directory to your PATH environment variable, check it. Be sure it is not out-of-date.
One more setup item. Selenium creates project files that have a SIDE file extension. I used a directory outside of my test environment. So, in my test environment I have to specify the full path to the *.side files. I suggest creating a directory in your test environment and copying the file to that directory. Then if you need to make changes you still have the original working script for Selenium IDE.
Once the setup is complete you can run the Selenium script from the command prompt in your test environment. The command format is:
selenium-side-runner <full path>/Salesforce.side
There are some other command options documented at the website. But this is a good start to verify your environment is operational for Selenium command line testing.
CAVEAT: A script generated in Selenium IDE may not execute with the same success in the Selenium Side Runner version. I had to disable a Mouse out test that thankfully was generated but not required. You may experience other issues like that.
Selenium IDE Exported Code
As I mentioned earlier, I generated my sample Salesforce script using the Python Pytest option. I was happy that the code came with comments and all the library references. But the execution failed.
I was able to get the script to work with some modifications. But the code actions are not working. Four lines of code failed. Two of the lines of code are not necessary. They cause mouse over and mouse out actions. I can live without them. But the other two lines of code should cause the View profile icon to open a logout popup, and then the last line of code should click the LOGOUT link.
This logout process is not happening. For now I have added code to handle the error conditions and then close the browser which ends the Salesforce session. Some applications must have logout. Otherwise, the session remains open. My habit is to test logout.
Here is a sample of the code I generated using Selenium IDE and converted to Python Pytest. The code generated and later modified is long, but useful if you want to test Salesforce or another web application that has login and logout functionality.
| # Generated by Selenium IDE |
| import os |
| from dotenv import load_dotenv |
| import time |
| from selenium.common.exceptions import NoSuchElementException |
| from selenium import webdriver |
| from selenium.webdriver.common.by import By |
| from selenium.webdriver.common.action_chains import ActionChains |
| from webdriver_manager.chrome import ChromeDriverManager |
| # This a suggested syntax for executing the Python Pytest script # pytest pwtests/test_salesforceLoginLogout.py -s –headed –browser chromium –tracing on |
| class TestSalesforceLoginLogout(): |
| def setup_method(self, method): |
| self.driver = webdriver.Chrome(ChromeDriverManager().install()) |
| # self.driver = webdriver.Chrome() |
| self.vars = {} |
| def teardown_method(self, method): |
| self.driver.quit() |
| x = 1 |
| def build_profile_list(self): |
| # Load Environment Variables and Populate local variables |
| load_dotenv() |
| my_id = os.environ.get(“uNAME”) |
| my_secret_key = os.environ.get(“pWORD”) |
| my_app = os.environ.get(“sAPP”) |
| # Build CSS Table |
| css_list = [] |
| css_list.append(“.profileTrigger > .uiImage”) |
| css_list.append(“#oneHeader > div.slds-global-header.slds-grid.slds-grid–align-spread > span > “ |
| “div.slds-global-header__item > ul > li:nth-child(9) > span > button”) |
| css_list.append(“#\38 8\:221\;a”) |
| # Build XPATH Table |
| xpath_list = [] |
| xpath_list.append(“//div[@id=’88:221;a’]/span/div/span”) |
| xpath_list.append(“//button/div/span/div/span”) |
| xpath_list.append(“//*[@id=’oneHeader’]/div[2]/span/div[2]/ul/li[9]/span/button”) |
| return css_list, xpath_list, my_id, my_secret_key, my_app |
| def test_salesforceLoginLogout(self): |
| css_list, xpath_list, my_id, my_secret_key, my_app = self.build_profile_list() |
| # Test name: Salesforce LoginLogout |
| # Step # | name | target | value | comment |
| # 1 | open | / | | |
| self.driver.get(my_app) |
| # 2 | setWindowSize | 1052×564 | | |
| self.driver.set_window_size(1052, 564) |
| # 3 | click | id=username | | |
| self.driver.find_element(By.ID, “username”).click() |
| # 4 | type | id=username | [email protected] | |
| self.driver.find_element(By.ID, “username”).send_keys(my_id) |
| # 5 | click | id=password | | |
| self.driver.find_element(By.ID, “password”).click() |
| # 6 | type | id=password | Winter01#2023 | |
| self.driver.find_element(By.ID, “password”).send_keys(my_secret_key) |
| # 7 | click | id=Login | | |
| self.driver.find_element(By.ID, “Login”).click() |
| # Look for popup alert (Simple, Prompt, Confirmation) |
| try: |
| # Switch the control to the Alert window |
| obj = self.driver.switch_to.alert |
| # Retrieve the message on the Alert window |
| msg = obj.text |
| print(“Alert shows following message: ” + msg) |
| time.sleep(2) |
| # use the accept() method to accept the alert |
| obj.accept() |
| print(” Clicked on the Accept Button in the Alert Window”) |
| except: |
| print(“Popup Alert not found”) |
| # Test for another popup alert |
| # 8 | mouseOver | css=.photoContainer | | |
| try: |
| element = self.driver.find_element(By.CSS_SELECTOR, “body > div.desktop.container.forceStyle.oneOne.” |
| “navexDesktopLayoutContainer.lafAppLayoutHost.forc” |
| “eAccess.tablet > div.viewport > section > div.oneUtilityBarContainer” |
| “.oneUtilityBar > div.slds-docked_container.forceDockingPanel” |
| “–xlarge.forceDockingPanel–scrollable.forceDockingPanel.DOCKED” |
| ” > div > div > div > a > div > div.slds-col–bump-left > span:nth-child(3) > button”) |
| actions = ActionChains(self.driver) |
| actions.move_to_element(element).perform() |
| print(“Mouse over Photo Container passed”) |
| except NoSuchElementException: |
| print(“NoSuchElementException:”, ” Failed to Mouse Over Photo Container”) |
| pass |
| # 9 | mouseOut | css=.photoContainer | | |
| try: |
| element = self.driver.find_element(By.CSS_SELECTOR, “body”) |
| actions = ActionChains(self.driver) |
| #actions.move_to_element(element, 0, 0).perform() |
| actions.move_to_element(element).perform() |
| print(“Body Locate passed”) |
| except NoSuchElementException: |
| print(“NoSuchElementException:”, ” Failed to locate body”) |
| pass |
| for x in css_list: |
| # 10 | click | css=.profileTrigger > .uiImage | | |
| try: |
| self.driver.find_element(By.CSS_SELECTOR, x).click() |
| #self.driver.find_element(By.XPATH, “”).click() |
| print(“View Profile CSS Locator passed for “+x) |
| except NoSuchElementException: |
| print(“NoSuchElementException:”, ” Failed to locate profileTrigger: “+x) |
| pass |
| for x in xpath_list: |
| # 10 | click | css=.profileTrigger > .uiImage | | |
| try: |
| #self.driver.find_element(By.CSS_SELECTOR, x).click() |
| self.driver.find_element(By.XPATH, x).click() |
| print(“View Profile XPath Locator passed for “+x) |
| except NoSuchElementException: |
| print(“NoSuchElementException:”, ” Failed to locate profileTrigger “+x) |
| pass |
| # 11 | click | linkText=Log Out | | |
| try: |
| self.driver.find_element(By.LINK_TEXT, “Log Out”).click() |
| print(“Logout Link Locator passed”) |
| except NoSuchElementException: |
| print(“NoSuchElementException:”, ” Failed to locate Log Out text”) |
| pass |
| # 12 | close | | | |
| self.driver.close() |
This code example is a true Pytest executable class. The code includes setup and teardown functions. It includes the main test function. And I added a function that builds list arrays and captures environment variables holding sensitive security data. I also added some TRY/EXCEPT code along with PRINT statements to handle error conditions. And to report what locators are working.
I wish I could tell you that the locator code works consistently. But I cannot. When I run the test repeatedly, I do not get the same results. Sometimes one or two locators work, and other times no locator processing works. The sample I included here is an example of when most of the locators worked as expected.
The common Python error condition you will see working with locators is the NoSuchElementException. I have a total of nine locator tests in this class. Three failed. That is not bad, but I would like it to be better.
Closing Remarks
I trust that you found this article useful. I hope you can see that Playwright is not a total replacement for Selenium. Though I still see issues with using Selenium. Both tools are strong, and people are dedicated to improving how they interface with languages and other test tools.
Here are some good points for deciding to use one tool versus the other:
What can we say about Playwright as a test tool and test interface?
-
-
- Cross-browser testing: Playwright supports testing on multiple browsers including Edge, Chrome, Firefox, and Safari, allowing you to easily perform cross-browser testing without the need to switch between different tools.
- Speed and performance: Playwright is designed to be fast and efficient. The Codegen tool that comes with Playwright is a powerful script recording tool. It can write language code recording user navigation through a target application. It’s possible to run tests quickly and handle complex and heavy-duty testing scenarios.
- Easy to use: Playwright has a modern and intuitive API. It’s easy to write tests and maintain the test scripts, even for non-technical users.
- Advanced features: Playwright offers advanced features like automatic wait and retry, support for headless browsing, and built-in screenshot, video, and PDF generation, making it a powerful tool for automation testing.
-
Does that not also apply to Selenium?
-
-
- Widespread use: Selenium has been widely adopted by the automation testing community, making it a well-established and widely supported tool.
- Support for multiple programming languages: Selenium supports a wide range of programming languages. Two of which are not supported by Playwright – C#, and Ruby. This increases its language support over Playwright.
- Ease of use: It may have not started out as easy, but it is getting easier to use because of the community, time it’s been on the market, and new Selenium IDE.
- Large community: Selenium has a large and active community of users, making it easy to find support and resources when needed.
- Support for Selenium Grid: Selenium Grid allows you to run tests on multiple machines, making it possible to scale testing efforts as needed.
-
Each of these tools has its own strengths and weaknesses, and the best choice will depend on the specific requirements of your automation testing effort. In general, Playwright is a good choice for projects that require fast and efficient testing, while Selenium is a good choice for projects that require support for other programming languages and a large community of users.