aw-playwright-demo v0.0.1

aw playwright-template

This is a template for those getting started with Playwright. It contains some additional tooling to make it easier to debug tests and make it easier to run non-default web applications and microfrontends in the test framework.

The website under test is a sample website built by Saucelabs called the-internet. It's located at https://the-internet.herokuapp.com. I also maintain a copy at https://the-internet.onrender.com, and the one hosted at onrender.com is used in most of the tests.

Installation

You need to configure npm to pull private packages from the @adaptavant scope. See the section in internal-services - git-registry-configuration - To authenticate in CICD for instructions on configuring $HOME/.npmrc.

Once configured, install modules with the following command:

$ npm install

We recommend Google Secret Manager for managing secrets. However, this template doesn't have the @adaptavant/secrets-manager package installed. Instead, perhaps the easiest way to get started writing tests when there are credentials or other secrets to manage is to use a .env file. This file is already added to .gitignore so that git won't see it as an untracked file. It's important to avoid committing secrets. To create the .env file, copy sample.env to .env:

$ cp sample.env .env

Then edit .env to replace the USERNAME and PASSWORD with actual values. For the-internet, you can find the test username and password on the Form Authentication page.

Once the .env file is created, you can move to the next section and run the tests.

In the future, once you're setup with a Google Cloud Platform project, you can work with IT to configure Secret Manager for your project. .env files are good for getting started as a solo engineer, but once there are two or more people involved, it becomes difficult to share and manage secrets safely and securely.

Run tests

Tests run against https://the-internet.onrender.com, a site for QA engineers to practice writing tests against a fake website.

$ npm run test:login

When done, two reports are generated:

• playwright-report: This is the default Playwright reporter. It's called html in playwright.config.ts.
• allure-report: This is the Allure report, using the allure-playwright module.

There is also commented code for the playwright-qase-reporter, which uploads test results to Qase. You need a Qase API token in order for the reporter to write test results to your Qase project.

Running Tests in CI

Capturing Test Results

This also includes the @adaptavant/playwright-push-to-google-sheet reporter. This reporter is intended to be used in GitHub Actions. It captures one of the HTML test reports, as well as test result data, uploads the HTML report to the https://testing-reports.setmore.com secure server, and uploads the raw totals and report URL to the Setmore 2.0 Test Automation Master Sheet.

It requires the WORKFLOWS_TOKEN, which is available by default in all Adaptavant GitHub workflows, as well as a projectCode and a spreadsheetId. See Save Test Execution Results in Test Master Sheet for more details, as well as alternatives to the reporter in CI, using two GitHub Custom Actions instead.

To run the tests in CI, we need to configure the reporter to use the correct values:

 ['@adaptavant/playwright-push-to-google-sheet-reporter', {
      projectCode: 'DP',  // Qase projectCode or repository name
      githubToken: process.env.WORKFLOWS_TOKEN,
      logger: logger,     // use detailed-logger for better debugging
      "remoteProjectFolder": "Setmore",   // Supplied by IT
      "serverUsername": "test",           // Supplied by IT
      "reportServerName": "127.0.0.1",    // Supplied by IT
      "localReportPath": "./playwright-report",   // local foldername for test report
      "keypath": `path/to/secrets/report.ssh`     // Supplied by IT
}]

Running tests on non-default versions

This template also demonstrates how to run tests on non-default versions of the application using the @adaptavant/environment-detector module. The environment-detector dynamically detects the environment based on a configuration file:

{
  "live": [
      "https://the-internet.onrender.com/"
  ],
  "staging": [
      "https://the-internet-[\\w-]*.onrender.com/"  
  ],
  "heroku": [
    "https://the-internet.herokuapp.com"
  ],
  "ossnet": [
      "https://ossnet.alwaysdata.net"
  ]
}

From the config file, we see that https://the-internet.onrender.com/ is the live environment. To deal with non-default versions, the environment-detector uses regular expression pattern matchers to identify an infinite number of possible URLs that would match staging. One such example is a pull request I opened on the-internet. These changes are visible on the non-default URL, https://the-internet-pr-1.onrender.com/. The pr-1 matches the portion of the URL matcher for the staging environment. We also add two additional named environments, one for Heroku, and one for ossnet.

To run tests on different environments, we pass in the URL as a BASEURL environment variable. This gets injected into the test environment so that all tests run against the injected URL, dynamically, without changing any code.

Try it for yourself with this command:

BASEURL=https://the-internet-pr-1.onrender.com npm run test:login

Note the following log messages:

playwright.config.ts:10: <info>: Environment is NOT live
playwright.config.ts:15: <info>: BASEURL = https://the-internet-pr-1.onrender.com

Auto-detection of the environment using the dynamically injected BASEURL also allows us to identify tests which should only run in specific environments.

if(environment.isNotLive())  // only run this test if it's not the live environment
  test('unsuccessful login with a bad username', async ({ page }) => {
    await page.goto('/login');
    await page.waitForSelector('#username');
    await page.type('#username', process.env.USERNAME + 'this_is_invalid_username');
    await page.type('#password', process.env.PASSWORD);
    await page.click('button[type="submit"]');
    await expect(page.getByText('Your username is invalid!')).toBeVisible();
  });

We can also identify specific environments

if(environment.isHeroku()) {
    logger.warn('We are running tests on Sauce Labs instance of the-internet, not the on-render instance!');
}

We can also control timeouts. Many of our staging environments are significantly slower than production, so we can keep minimal waits in production, while expanding them only for staging:

/* Maximum time one test can run for. */
  timeout: environment.isLive() 
    ? 30 * 1000  // live
    : 60 * 1000,  // not live
  expect: {
    /**
     * Maximum time expect() should wait for the condition to be met.
     * For example in `await expect(locator).toHaveText();`
     */
    timeout: environment.isLive()
      ? 5000    // live
      : 15000   // not live
  },

Perhaps the biggest advantage to using the environment-detector is that we can connect any development non-default deployment to the test runner in CI. For instance, if I push changes to the pr-1 instance of the-internet, we can take that URL and pass it in as the BASEURL when executing the tests in CI. This helps us shift left in testing.

  - name: Run tests
    run: |
      export BASEURL=${{ inputs.baseurl }}   # inputs.baseurl is from workflow_dispatch trigger.
      npx playwright test

Logging

This template uses a logging module called detailed-logger. It adds additional context to the logs, such as the filename and line number. This is incredibly helpful for debugging purposes.

playwright.config.ts:10: <info>: Environment is NOT live
playwright.config.ts:15: <info>: BASEURL = https://the-internet-pr-1.onrender.com
push-to-google-sheet-reporter.js:21: <warn>: Not running in CI. Disabling push-to-google-sheet-reporter.

Although not visible in the README, the "info" messages are displayed in green, while "warn" messages are displayed in yellow. Also, we see the BASEURL entry is ommitted from line 15 of playwright.config.ts.

Unit tests

This template also demonstrates how to write unit tests for framework level functionality you may write as a developer or tester working on writing tests.

$ npm run unit-tests

This uses Jasmine to run unit tests in the unit-tests folder. The tests validate that the environment-detector configuration file is setup correctly for the 4 environments. Since they're unit tests, we use Jasmine instead of Playwright, and because Jasmine uses soft assertions.

Static code analysis with eslint

This template is configured with eslint for both static code analysis and formatting and auto-fixing issues. To see issues, run the following command:

$ npm run eslint

To automatically fix them, use this command instead:

$ npm run eslint:fix

Import aliases

Import aliases allow you to configure absolute import paths instead of relative ones. This takes this:

import somePageObject from '../../page-objects/some-page-object.ts';

And turns it into this:

import somePageObject from '#@/page-objects/some-page-object.ts';

We configure an alias to the root of the project in 3 places:

tsconfig.json - So VSCode can resolve paths:

        "paths": {
            "#@/*": [
                "./*"
            ]
        }

As well as in package.json, so node.js and playwright can resolve paths:

  "imports": {
    "#@/*": "./*"
  },

And in .eslintrc, so eslint can resolve the paths:

    "settings": {
        "import/resolver": {
            "eslint-import-resolver-custom-alias": {
                "alias": {
                    "#@": "./"
                }
            }
        }
    },

This helps make refactoring and restructuring the project a little easier, since the import paths are the same regardless of where the files are located relative to one another.

Setting up CI

This template comes with the default workflow, playwright.yml, setup by Playwright when it's installed. It won't work out of the box if you have secrets to manage. To manage secrets, we recommend Google Secret Manager, and we have a client library @adaptavant/secrets-manager.

There's also an example of running Playwright tests in parallel using GitHub Actions and Playwright sharding. See playwright-parallel-demo for an example. It also won't work out of the box if you have secrets to manage, but it can serve as a reference for how to take test result data from various workers and combine them into a single report.

Why TypeScript?

TypeScript is a superset of JavaScript. It adds static typing that can help catch errors in code early instead of at runtime. What's more, since it's a superset of JavaScript, all of the JavaScript packages from the Node.js ecosystem are compatible with TypeScript. Moreover, JavaScript and TypeScript can both be used in a TypeScript project.

Contributing

If you'd like to fix a bug or add functionality, first, create an issue in the issue tracker for discussion. Pull requests are accepted as long as the functionality doesn't cause breaking changes in the existing functionality or disturb the existing default options.

To contribute, clone the repository on your local workspace:

$ git clone https://github.com/Adaptavant/playwright-template.git

Create a branch using the issue number:

$ git checkout -b ANYWHERE-0/feature/is_1

Next, be sure to set your username and email address to be associated with your GitHub profile:

$ git config user.email "<your_email>@anywhere.co" && git config user.name "<YOUR_NAME>"