@khulnasoft/github-action NPM

khulnasoft/github-action

GitHub Action for running Khulnasoft end-to-end and component tests. Includes npm, pnpm and Yarn installation, custom caching and lots of configuration options.

Placing use: khulnasoft/github-action@v6 into a GitHub Action workflow gives you a simple way to run Khulnasoft. The action takes the project's npm, pnpm or Yarn package manager lock file, installs dependencies and caches these dependencies for future use. It then proceeds to run Khulnasoft end-to-end tests with the built-in Electron browser and provides a test summary after completion.

If you are testing against a running server like the Khulnasoft Kitchen Sink showcase example on https://example.khulnasoft.com/ no other parameters are necessary. In other cases where you need to fire up a development server, you can add the start parameter to the workflow. Browse through the examples to find other useful parameters.

Examples

End-to-End testing
Component testing
Select action version
Run tests in a given browser
- using Chrome
- using Firefox
- using Edge
- using headed mode
Using Docker image
Specify environment variables
Run only some spec files
Test project in subfolder
Record results on Khulnasoft Cloud
- Storing the Project ID and Record Key
- Getting Git information environment variables
- Getting PR and URL automatically
- Overwriting Merge SHA into SHA message
Tag recordings
Specify auto cancel after failures
Store test artifacts on GitHub
Quiet output
Set Khulnasoft config values
Use specific config file
Run tests in parallel
Combine Component and E2E testing
Build app before running the tests
Start server before running the tests
Start multiple servers before running the tests
Wait for server to respond before running the tests
wait-on with Node.js 18+ workarounds
Use custom install command
Use command prefix
Use own custom test command
Pass custom build id when recording to Khulnasoft Cloud
Generate a robust custom build id to allow re-running the workflow
Use different working-directory
Use subfolders
Use pnpm
Use Yarn Classic
Use Yarn Modern
Use Yarn Plug'n'Play
Use Yarn workspaces
Use custom cache key
Run tests on multiple Node versions
Split install and tests into separate jobs
Use custom install commands
Install only Khulnasoft to avoid installing all dependencies
Use timeouts to avoid hanging CI jobs
Print Khulnasoft info like detected browsers
Run tests nightly or on any schedule
Specify job summary title
Suppress job summary
More examples

Examples contained in this repository, based on current Khulnasoft versions, can be found in the examples directory. Examples for Legacy Configuration, which use Khulnasoft 9.7.0, are no longer maintained. They can be referred to in the examples/v9 directory of the v5 branch.

Live examples, such as example-basic.yml are shown together with a status badge. Click on the status badge to read the source code of the workflow, for example

Older external examples based on a Legacy Configuration for Khulnasoft 9 and earlier can be found in the README for version v5.

Note: this package assumes that khulnasoft is declared as a development dependency in the package.json file. The khulnasoft npm module is required to run Khulnasoft via its Module API.

End-to-End Testing

name: End-to-end tests
on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      # Install npm dependencies, cache them correctly
      # and run all Khulnasoft tests
      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6

The workflow file example-basic.yml shows how Khulnasoft runs on GH Actions using Ubuntu (20 and 22), Windows, and macOS without additional OS dependencies necessary.

This workflow uses the default test type of End-to-End (E2E) Testing. Alternatively, Component Testing can be utilized by referencing the Component Testing section below.

Component Testing

To use Khulnasoft Component Testing add component: true

name: End-to-end tests
on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          component: true

See the example project component-tests and the example-component-test.yml workflow for more details.

Action version

Best practice:

Our examples specify using branch v6 which is the action's recommended major version:

- name: Khulnasoft run
  uses: khulnasoft/github-action@v6

When using khulnasoft/github-action@v6 from your workflow file, you will automatically use the latest tag from branch v6.

Alternatively, to mitigate unforeseen breaks, bind to a specific tag, for example:

- name: Khulnasoft run
  uses: khulnasoft/github-action@v6.1.0

The changes associated with each tag are shown under GitHub's releases list. Refer also to the CHANGELOG for an overview of major changes.

Browser

Specify the browser name or path with the browser parameter. The default browser, if none is specified, is the built-in Electron browser.

Chrome

name: Chrome
on: push
jobs:
  chrome:
    runs-on: ubuntu-22.04
    name: E2E on Chrome
    steps:
      - uses: actions/checkout@v4
      - uses: khulnasoft/github-action@v6
        with:
          browser: chrome

Firefox

name: Firefox
on: push
jobs:
  firefox:
    runs-on: ubuntu-22.04
    name: E2E on Firefox
    steps:
      - uses: actions/checkout@v4
      - uses: khulnasoft/github-action@v6
        with:
          browser: firefox

Edge

name: Edge
on: push
jobs:
  edge:
    runs-on: ubuntu-22.04
    name: E2E on Edge
    steps:
      - uses: actions/checkout@v4
      - uses: khulnasoft/github-action@v6
        with:
          browser: edge

Headed

Run the browser in headed mode - as of Khulnasoft v8.0 the khulnasoft run command executes tests in headless mode by default

name: Chrome headed
on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    steps:
      - uses: actions/checkout@v4
      - uses: khulnasoft/github-action@v6
        with:
          browser: chrome
          headed: true

Docker image

You can run the action in a Docker container.

name: Test in Docker
on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    # Khulnasoft Docker image from https://hub.docker.com/r/khulnasoft
    # with browsers pre-installed
    container:
      image: khulnasoft/browsers:latest
      options: --user 1001
    steps:
      - uses: actions/checkout@v4
      - uses: khulnasoft/github-action@v6
        with:
          browser: chrome

Replace the latest tag with a specific version image tag from khulnasoft/browsers on Docker Hub to avoid breaking changes when new images are released (especially when they include new major versions of Node.js).

Include options: --user 1001 to avoid permissions issues.

Refer to khulnasoft/khulnasoft-docker-images for further information about using Khulnasoft Docker images. Khulnasoft offers the Khulnasoft Docker Factory to generate additional Docker images with selected components and versions.

Env

Specify the env argument with env parameter

name: Khulnasoft tests
on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Khulnasoft run with env
        uses: khulnasoft/github-action@v6
        with:
          env: host=api.dev.local,port=4222

When passing the environment variables this way, unfortunately due to GitHub Actions syntax, the variables should be listed in a single line, which can be hard to read. As an alternative, you can use the step's env block where every variable can be set on its own line. In this case, you should prefix every variable with KHULNASOFT_ because such variables are loaded by Khulnasoft automatically. The above code example is equivalent to:

name: Khulnasoft tests
on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Khulnasoft run with env
        uses: khulnasoft/github-action@v6
        env:
          KHULNASOFT_host: api.dev.local
          KHULNASOFT_port: 4222

For more examples, see the workflows below, using environment variables for recording.

Specs

Specify the spec files to run with spec parameter

name: Khulnasoft tests
on: push
jobs:
  khulnasoft-run:
    name: Khulnasoft run
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          spec: khulnasoft/e2e/spec1.cy.js

You can pass multiple specs and wild card patterns using multi-line parameter, see example-config.yml:

spec: |
  khulnasoft/e2e/spec-a.cy.js
  khulnasoft/**/*-b.cy.js

For more information, visit the Khulnasoft command-line docs.

Project

Specify the project to run with project parameter

name: Khulnasoft tests
on: push
jobs:
  khulnasoft-run:
    name: Khulnasoft run
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          project: ./some/nested/folder

For more information, visit the Khulnasoft command-line docs.

Record test results on Khulnasoft Cloud

By setting the parameter record to true, you can record your test results into the Khulnasoft Cloud. Read the Khulnasoft Cloud documentation to learn how to sign up and create a Khulnasoft Cloud project.

We recommend passing the GITHUB_TOKEN secret (created by the GH Action automatically) as an environment variable. This will allow correctly identifying every build and avoid confusion when re-running a build.

name: Khulnasoft tests
on: push
jobs:
  khulnasoft-run:
    name: Khulnasoft run
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          record: true
        env:
          # pass GitHub token to allow accurately detecting a build vs a re-run build
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Project ID and Record Key

To record the project needs projectId and recordKey.

Typically, the projectId is stored in the Khulnasoft Configuration File, while the recordKey is set as a CLI parameter. If you want to avoid this, both the projectId and recordKey can be provided as environment variables using GitHub secrets.

name: Khulnasoft tests
on: push
jobs:
  khulnasoft-run:
    name: Khulnasoft run
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          record: true
        env:
          # pass the Khulnasoft Cloud record key as an environment variable
          KHULNASOFT_RECORD_KEY: ${{ secrets.KHULNASOFT_RECORD_KEY }}
          # pass the project ID from the secrets through environment variable
          KHULNASOFT_PROJECT_ID: ${{ secrets.PROJECT_ID }}

Git information

Khulnasoft uses the @khulnasoft/commit-info package to associate Git details (branch, commit message, author) with each run. It typically uses Git commands, expecting a .git folder. In Docker or similar environments where .git is absent, or if you need different data in the Khulnasoft Cloud, Git information can be passed via custom environment variables.

name: Khulnasoft tests
on: push
jobs:
  khulnasoft-run:
    name: Khulnasoft run
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          record: true
        env:
          # Get the short ref name of the branch that triggered the workflow run
          COMMIT_INFO_BRANCH: ${{ github.ref_name }}

Please refer to the Khulnasoft Cloud Git information environment variables section in our documentation for more examples.

Please refer to the default GitHub environment variables for additional GitHub examples.

Automatic PR number and URL detection

When recording runs to Khulnasoft Cloud, the PR number and URL can be automatically detected if you pass GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} via the workflow env. When set, this value enables the Action to perform additional logic that grabs the related PR number and URL (if they exist) and sets them in the environment variables KHULNASOFT_PULL_REQUEST_ID and KHULNASOFT_PULL_REQUEST_URL, respectively.

See Khulnasoft' documentation on CI Build Information

Example workflow using the variables:

name: Example echo PR number and URL
on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          record: true
      - run: echo "PR number is $KHULNASOFT_PULL_REQUEST_ID"
      - run: echo "PR URL is $KHULNASOFT_PULL_REQUEST_URL"
    env:
      KHULNASOFT_RECORD_KEY: ${{ secrets.KHULNASOFT_RECORD_KEY }}
      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Triggering event: `pull_request`/`pull_request_target`

For either of these events, we set KHULNASOFT_PULL_REQUEST_ID and KHULNASOFT_PULL_REQUEST_URL to that of the PR number and URL, respectively, of the PR that triggered the workflow.

Triggering event: `push`

When a commit on a branch without a PR is made, the Khulnasoft GitHub Action checks to see if the commit that triggered the workflow has a related PR. If the commit exists in any other PRs, it's considered a related PR. When there are related PRs, we grab the first related PR and use that PR's number and URL for KHULNASOFT_PULL_REQUEST_ID and KHULNASOFT_PULL_REQUEST_URL, respectively.

If no related PR is detected, KHULNASOFT_PULL_REQUEST_ID and KHULNASOFT_PULL_REQUEST_URL will be undefined.

Merge SHA into SHA

We recommend using the action with on: push rather than on: pull_request or on: merge_group for more accurate commit information in Khulnasoft Cloud. When running on pull_request or merge_group, the commit message defaults to "merge SHA into SHA". You can overwrite the commit message sent to Khulnasoft Cloud by setting an environment variable.

name: Khulnasoft tests
on: push
jobs:
  khulnasoft-run:
    name: Khulnasoft run
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          record: true
        env:
          # overwrite commit message sent to Khulnasoft Cloud
          COMMIT_INFO_MESSAGE: ${{github.event.pull_request.title}}
          # re-enable PR comment bot
          COMMIT_INFO_SHA: ${{github.event.pull_request.head.sha}}

See issue 124 for details.

Tag recordings

You can pass a single or multiple tags when recording a run. For example

name: tags
on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    # let's make sure our "app" works on several versions of Node
    strategy:
      matrix:
        node: [18, 20, 21]
    name: E2E on Node v${{ matrix.node }}
    steps:
      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node }}
      - run: node -v

      - name: Checkout
        uses: actions/checkout@v4

      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          record: true
          tag: node-${{ matrix.node }}
        env:
          KHULNASOFT_RECORD_KEY: ${{ secrets.KHULNASOFT_RECORD_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

The recording will have tags as labels on the run.

Specify auto cancel after failures

Specify the number of failed tests that will cancel a run when using the Khulnasoft Cloud Auto Cancellation feature.

This feature requires Khulnasoft 12.6.0 or later and a Khulnasoft Cloud Business or Enterprise account.

name: Khulnasoft E2E Tests
on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    name: E2E
    steps:

      - name: Checkout
        uses: actions/checkout@v4

      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          record: true
          # Cancel the run after 2 failed tests
          auto-cancel-after-failures: 2
        env:
          KHULNASOFT_RECORD_KEY: ${{ secrets.KHULNASOFT_RECORD_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

See Auto Cancellation for more information.

Artifacts

If you don't record the test run on Khulnasoft Cloud, you can still store generated videos and screenshots as CI artifacts. See the workflow example below.

name: Artifacts
on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    name: Artifacts
    steps:
      - uses: actions/checkout@v4
      - uses: khulnasoft/github-action@v6
      # after the test run completes store videos and any screenshots
      - uses: actions/upload-artifact@v4
        # add the line below to store screenshots only on failures
        # if: failure()
        with:
          name: khulnasoft-screenshots
          path: khulnasoft/screenshots
          if-no-files-found: ignore # 'warn' or 'error' are also available, defaults to `warn`
      - uses: actions/upload-artifact@v4
        with:
          name: khulnasoft-videos
          path: khulnasoft/videos
          if-no-files-found: ignore # 'warn' or 'error' are also available, defaults to `warn`

Quiet flag

You can provide quiet flag for khulnasoft run to silence any Khulnasoft specific output from stdout

name: example-quiet
on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      # Install npm dependencies, cache them correctly
      # and run all Khulnasoft tests with `quiet` parameter
      - name: Khulnasoft run
        uses: ./
        with:
          working-directory: examples/quiet
          quiet: true

Config

Specify configuration values with config parameter

name: Khulnasoft tests
on: push
jobs:
  khulnasoft-run:
    name: Khulnasoft run
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          config: pageLoadTimeout=100000,baseUrl=http://localhost:3000

Config File

Specify the path to your Configuration File with config-file parameter

name: Khulnasoft tests
on: push
jobs:
  khulnasoft-run:
    name: Khulnasoft run
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          config-file: khulnasoft.config-alternate.js

Parallel

Note: Khulnasoft parallelization requires a Khulnasoft Cloud account.

You can spin multiple containers running in parallel using strategy: matrix argument. Just add more dummy items to the containers: [1, 2, ...] array to spin more free or paid containers. Then use record and parallel parameters to load balance tests.

name: Parallel Khulnasoft Tests
on: push
jobs:
  test:
    name: Khulnasoft run
    runs-on: ubuntu-22.04
    strategy:
      # when one test fails, DO NOT cancel the other
      # containers, because this will kill Khulnasoft processes
      # leaving Khulnasoft Cloud hanging ...
      # https://github.com/khulnasoft/github-action/issues/48
      fail-fast: false
      matrix:
        # run 3 copies of the current job in parallel
        containers: [1, 2, 3]
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      # because of "record" and "parallel" parameters
      # these containers will load balance all found tests among themselves
      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          record: true
          parallel: true
          group: 'Actions example'
        env:
          # pass the Khulnasoft Cloud record key as an environment variable
          KHULNASOFT_RECORD_KEY: ${{ secrets.KHULNASOFT_RECORD_KEY }}
          # Recommended: pass the GitHub token lets this action correctly
          # determine the unique run id necessary to re-run the checks
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Parallel run

Warning ⚠️: Khulnasoft actions use GITHUB_TOKEN to get the correct branch and the number of jobs run, making it possible to re-run without the need of pushing an empty commit. If you don't want to use the GITHUB_TOKEN you can still run your tests without problem with the only note that Khulnasoft Cloud's API connects parallel jobs into a single logical run using GitHub commit SHA plus workflow name. If you attempt to re-run GitHub checks, Khulnasoft Cloud thinks the run has already ended. In order to truly rerun parallel jobs, push an empty commit with git commit --allow-empty -m "re-run checks" && git push. As another work around you can generate and cache a custom build id, read Adding a unique build number to GitHub Actions.

The Khulnasoft GH Action does not spawn or create any additional containers - it only links the multiple containers spawned using the matrix strategy into a single logical Khulnasoft Cloud run where it splits the specs amongst the machines. See the Khulnasoft parallelization guide for the explanation.

During staged rollout of a new GitHub-hosted runner version, GitHub may provide a mixture of current and new image versions used by the container matrix. It is recommended to use a Docker image in the parallel job run which avoids any Khulnasoft Cloud errors due to browser major version mismatch from the two different image versions. A Docker image is not necessary if testing against the default built-in Electron browser because this browser version is fixed by the Khulnasoft version in use and it is unaffected by any GitHub runner image rollout.

Component and E2E Testing

Component Testing and End-to-End (E2E) Testing types can be combined in the same job using separate steps

- name: Run E2E tests
  uses: khulnasoft/github-action@v6

- name: Run Component Testing
  uses: khulnasoft/github-action@v6
  with:
    # we have already installed everything
    install: false
    component: true

See the example project component-test and the example-component-test.yml workflow for more details.

Build app

You can run a build step before starting tests

name: Build
on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          build: npm run build

Start server

If your tests run against a local server, use the start parameter to start your server. The server will run in the background.

name: With server
on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          start: npm start

Caution: use the start parameter only to start a server, not to run Khulnasoft, otherwise tests may be run twice. The action runs Khulnasoft tests by default, unless the parameter runTests is set to false.

Note: sometimes on Windows you need to run a different start command. You can use the start-windows parameter for this.

name: With server
on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          # Linux and MacOS
          start: npm start
          # Takes precedences on Windows
          start-windows: npm run start:windows:server

Note: A server continues to run until the end of the GitHub workflow job that started it. At the end of the job the GitHub workflow runner executes a "Complete job" phase automatically where it terminates any server processes which are still running.

Start multiple servers

You can start multiple server processes. For example, if you have an API to start using npm run api and the web server to start using npm run web you can put those commands in start using comma separation.

name: With servers
on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          start: npm run api, npm run web

You can place the start commands in separate lines

with:
  start: |
    npm run api
    npm run web

Wait-on

If you are starting a local server and it takes a while to start, you can add a parameter wait-on and pass url to wait for the server to respond.

name: After server responds
on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          start: npm start
          # quote the url to be safe against YML parsing surprises
          wait-on: 'http://localhost:8080'

(also uses wait-on)

By default, wait-on will retry for 60 seconds. You can pass a custom timeout in seconds using wait-on-timeout.

- uses: khulnasoft/github-action@v6
  with:
    start: npm start
    wait-on: 'http://localhost:8080/status'
    # wait for 2 minutes for the server to respond
    wait-on-timeout: 120

You can wait for multiple URLs to respond by separating urls with a comma

- uses: khulnasoft/github-action@v6
  with:
    # API runs on port 3050
    # Web server runs on port 8080
    start: npm run api, npm run web
    # wait for all services to respond
    wait-on: 'http://localhost:3050, http://localhost:8080'

The action will wait for the first url to respond, then will check the second url, and so on.

You can even use your own command (usually by using npm, yarn, npx) to wait for the server to respond. For example, if you want to use the wait-on utility to ping the server and run the Khulnasoft tests after the server responds:

- uses: khulnasoft/github-action@v6
  with:
    start: npm start
    wait-on: 'npx wait-on --timeout 60000 http://localhost:3000'

See example-wait-on.yml workflow file.

If this action times out waiting for the server to respond, please see Debugging section in this README file.

`wait-on` with Node.js 18+

Under Node.js version 18 and later, wait-on may fail to recognize that a localhost server is running. This affects development web servers which do not listen on both IPv4 and IPv6 network stacks.

Check your server documentation to see if it can be started using 0.0.0.0 (all addresses) and use this if available. If this option is not available or does not resolve the issue then carry on to the next steps:
If the action log shows that wait-on is failing to connect to 127.0.0.1, replace localhost by [::1] (the IPv6 loopback address)
If the action log shows that wait-on is failing to connect to ::1, replace localhost by 127.0.0.1 (the IPv4 loopback address)

Custom install command

If you want to overwrite the install command

- uses: khulnasoft/github-action@v6
  with:
    install-command: yarn --frozen-lockfile --silent

See example-install-command.yml workflow file.

Command prefix

You can prefix the default test command using the command-prefix option. This is useful for example when running Percy, which requires the test command to be wrapped with percy exec --.

name: Visual
on: push
jobs:
  e2e:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          start: npm start
          # quote the url to be safe against YML parsing surprises
          wait-on: 'http://localhost:8080'
          # the entire command will automatically be prefixed with "npm"
          # and we need the second "npm" to execute "khulnasoft run ..." command line
          command-prefix: 'percy exec -- npx'

Custom test command

You can overwrite the Khulnasoft run command with your own command.

steps:
  - name: Checkout 🛎
    uses: actions/checkout@v4

  - name: Custom tests 🧪
    uses: khulnasoft/github-action@v6
    with:
      command: npm run e2e:ci

Caution: using the action parameter command causes multiple other parameters to be ignored including: auto-cancel-after-failures, browser, ci-build-id, command-prefix, component, config, config-file, env, group, headed, parallel, project, publish-summary, quiet, record, spec and tag.

See example-custom-command.yml file.

Custom build id

You can overwrite ci-build-id used to link separate machines running tests into a single parallel run.

name: Parallel
on: push
jobs:
  test:
    runs-on: ubuntu-22.04
    strategy:
      matrix:
        # run 3 copies of the current job in parallel
        containers: [1, 2, 3]
    steps:
      - uses: actions/checkout@v4
      - uses: khulnasoft/github-action@v6
        with:
          record: true
          parallel: true
          group: 'Actions example'
          ci-build-id: '${{ github.sha }}-${{ github.workflow }}-${{ github.event_name }}'
        env:
          # pass the Khulnasoft Cloud record key as an environment variable
          KHULNASOFT_RECORD_KEY: ${{ secrets.KHULNASOFT_RECORD_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Tip: see Learn GitHub Actions, with sections on Expressions, Contexts and Environment variables.

Robust custom build id

If you re-run the GitHub workflow, if you use the same custom build id during recording, Khulnasoft Cloud will cancel the run with "Build already finished" error. To avoid this, you need to generate a new custom build id on every workflow re-run. A good solution showing in the example-custom-ci-build-id.yml file is to run a common job first that just generates a new random ID. This ID can be used by the testing jobs to tie the build together. If the user re-runs the workflow a new unique build id is generated, allowing recording the new Khulnasoft Cloud run.

jobs:
  # single job that generates and outputs a common id
  prepare:
    outputs:
      uuid: ${{ steps.uuid.outputs.value }}
    steps:
      - name: Generate unique ID 💎
        id: uuid
        # take the current commit + timestamp together
        # the typical value would be something like
        # "sha-5d3fe...35d3-time-1620841214"
        run: echo "value=sha-$GITHUB_SHA-time-$(date +"%s")" >> $GITHUB_OUTPUT
  smoke-tests:
    needs: ['prepare']
    steps:
      - uses: actions/checkout@v4
      - uses: khulnasoft/github-action@v6
        with:
          record: true
          parallel: true
          ci-build-id: ${{ needs.prepare.outputs.uuid }}
        env:
          # pass the Khulnasoft Cloud record key as an environment variable
          KHULNASOFT_RECORD_KEY: ${{ secrets.EXAMPLE_RECORDING_KEY }}

See the example-custom-ci-build-id.yml for the full workflow.

Working directory

In a monorepo, the end-to-end or component test might be placed in a different sub-folder from the application itself. This sub-folder is the Khulnasoft "working directory" which you can specify using the working-directory parameter.

In the following example of a directory layout for end-to-end testing, the Khulnasoft working directory is app-test. The working directory contains the Khulnasoft tests and a package manager lock file:

repo/
  app/
  app-test/
    khulnasoft/
      e2e/
      fixtures/
      support/
    khulnasoft.config.js
  package.json
  package-lock.json

We use working-directory: app-test to match the above example directory structure:

on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    steps:
      - uses: actions/checkout@v4
      - uses: khulnasoft/github-action@v6
        with:
          start: npm start
          working-directory: app-test

See the Khulnasoft documentation Folder structure section for examples of standard directory layouts, covering end-to-end testing and component testing with both JavaScript and TypeScript options.

Each of the examples in this monorepo is separated from other examples by using different working directories. See example-basic.yml for one end-to-end test example using the parameter working-directory and example-component-test.yml for a component test example.

Subfolders

Sometimes the application under test and the Khulnasoft end-to-end tests may have separately defined dependencies. In the example below, Khulnasoft has its own package.json file in a subfolder:

root/
  e2e/
    (code for installing and running Khulnasoft tests)
    package.json
    package-lock.json
    khulnasoft.config.js
    khulnasoft/

  (code for running the "app" with "npm start")
  package.json
  package-lock.json

In this case you can first install the dependencies for the application (npm ci), then start the application server (npm start) before calling khulnasoft/github-action to install the dependencies for Khulnasoft and to run Khulnasoft. You may also need to use the wait-on parameter to make sure that the app server is fully available.

name: E2E
on: push
jobs:
  test:
    runs-on: ubuntu-22.04
    steps:
      - uses: actions/checkout@v4
      - name: Install root dependencies
        run: npm ci

      - name: Start server in the background
        run: npm start &

      # Khulnasoft has its own package.json in folder "e2e"
      - name: Install Khulnasoft and run tests
        uses: khulnasoft/github-action@v6
        with:
          working-directory: e2e

pnpm

The package manager pnpm is not pre-installed in GitHub Actions runner images (unlike npm and yarn): to install pnpm include pnpm/action-setup in your workflow. If the action finds a pnpm-lock.yaml file, it uses the pnpm command pnpm install --frozen-lockfile by default to install dependencies. At this time the action does not automatically cache dependencies installed by pnpm. The example below includes steps to locate the pnpm store directory and to cache its contents for later use.

name: example-basic-pnpm
on: push
jobs:
  basic-pnpm:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Install pnpm
        uses: pnpm/action-setup@v2
        with:
          version: 8
      - name: Get pnpm store directory
        shell: bash
        run: |
          echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_ENV
      - name: Setup pnpm cache
        uses: actions/cache@v3
        with:
          path: ${{ env.STORE_PATH }}
          key: ${{ runner.os }}-pnpm-store-${{ hashFiles('examples/basic-pnpm/pnpm-lock.yaml') }}
          restore-keys: |
            ${{ runner.os }}-pnpm-store-
      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          working-directory: examples/basic-pnpm

Yarn Classic

If a yarn.lock file is found, the action uses the Yarn 1 (Classic) command yarn --frozen-lockfile by default to install dependencies.

name: example-yarn-classic
on: push
jobs:
  yarn-classic:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          working-directory: examples/yarn-classic

Yarn Modern

To install dependencies using a yarn.lock file from Yarn Modern (Yarn 2 and later) you need to override the default Yarn 1 (Classic) installation command yarn --frozen-lockfile. You can do this by using the install-command parameter and specifying yarn install for example:

name: example-yarn-modern
on: push
jobs:
  yarn-modern:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          working-directory: examples/yarn-modern
          install-command: yarn install

This example covers the .yarnrc.yml configuration nodeLinker: node-modules which Yarn uses by default for projects updated from Yarn Classic. For nodeLinker: pnp see Yarn Plug'n'Play below. (Note that github-action is not compatible with the nodeLinker: pnpm setting.)

Yarn Plug'n'Play

When using Yarn Modern (Yarn 2 and later) with Plug'n'Play enabled, you will need to use the command parameter to run yarn instead of npx.

name: example-yarn-modern-pnp
on: push
jobs:
  yarn-classic:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          working-directory: examples/yarn-modern-pnp
          install-command: yarn install
          command: yarn run --binaries-only khulnasoft run

This example covers the .yarnrc.yml configuration when nodeLinker is undefined or set to nodeLinker: pnp corresponding to Yarn Plug'n'Play. Yarn uses this by default for projects newly created with Yarn Modern.

Caution: using the action parameter command causes multiple other parameters to be ignored. See command section for more information.

Yarn workspaces

This action should discover the Yarn workspaces correctly. For example, see folder examples/start-and-yarn-workspaces and workflow file example-start-and-yarn-workspaces.yml

name: example-start-and-yarn-workspaces
on: push
jobs:
  single:
    # the example has Yarn workspace in its "root" folder
    # examples/start-and-yarn-workspaces
    # and tests in a subfolder like "workspace-1"
    runs-on: ubuntu-22.04
    steps:
      - uses: actions/checkout@v4
      - uses: khulnasoft/github-action@v6
        with:
          working-directory: examples/start-and-yarn-workspaces/workspace-1
          build: yarn run build
          start: yarn start
          wait-on: 'http://localhost:5000'

Custom cache key

Sometimes the default cache key does not work. For example, if you cannot share the Node modules across Node versions due to native extensions. In that case pass your own cache-key parameter.

name: End-to-end tests
on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    # let's make sure our "app" works on several versions of Node
    strategy:
      matrix:
        node: [18, 20, 21]
    name: E2E on Node v${{ matrix.node }}
    steps:
      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node }}
      - name: Checkout
        uses: actions/checkout@v4
      # run Khulnasoft tests and record them under the same run
      # associated with commit SHA and just give a different group name
      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          record: true
          group: Tests on Node v${{ matrix.node }}
          cache-key: node-v${{ matrix.node }}-on-${{ runner.os }}-hash-${{ hashFiles('yarn.lock') }}
        env:
          KHULNASOFT_RECORD_KEY: ${{ secrets.KHULNASOFT_RECORD_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Node versions

You can run your tests across multiple Node versions.

name: Node versions
on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    strategy:
      matrix:
        node: [18, 20, 21]
    name: E2E on Node v${{ matrix.node }}
    steps:
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node }}
      - uses: actions/checkout@v4
      - uses: khulnasoft/github-action@v6

See the Node.js section for information about supported versions and usage of Node.js.

Split install and tests

Sometimes you may want to run additional commands between installation and tests. To enable this use the install and runTests parameters.

name: E2E
on: push
jobs:
  test:
    runs-on: ubuntu-22.04
    steps:
      - uses: actions/checkout@v4
      - name: Install dependencies
        uses: khulnasoft/github-action@v6
        with:
          # just perform install
          runTests: false
      - run: yarn lint
      - name: Run e2e tests
        uses: khulnasoft/github-action@v6
        with:
          # we have already installed all dependencies above
          install: false
          # Khulnasoft tests and config file are in "e2e" folder
          working-directory: e2e

See khulnasoft-gh-action-monorepo for a working example.

Custom install

Finally, you might not need this GH Action at all. For example, if you want to split the npm dependencies installation from the Khulnasoft binary installation, then it makes no sense to use this action. Instead you can install and cache Khulnasoft yourself.

Install Khulnasoft only

If the project has many dependencies, but you want to install just Khulnasoft you can combine this action with actions/cache and npm i khulnasoft commands yourself.

- uses: actions/checkout@v4
- uses: actions/cache@v2
  with:
    path: |
      ~/.cache/Khulnasoft
      node_modules
    key: my-cache-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
- run: npm i khulnasoft
- uses: khulnasoft/github-action@v6
  with:
    install: false

Timeouts

You can tell the CI to stop the job or the individual step if it runs for longer then a given time limit. This is a good practice to ensure the hanging process does not accidentally use up all your CI minutes.

jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    # stop the job if it runs over 10 minutes
    # to prevent a hanging process from using all your CI minutes
    timeout-minutes: 10
    steps:
      - uses: actions/checkout@v4
      - uses: khulnasoft/github-action@v6
        # you can specify individual step timeout too
        timeout-minutes: 5

More examples

Name	Description
khulnasoft/khulnasoft-example-kitchensink	Runs every API command in Khulnasoft using various CI platforms including GitHub Actions
khulnasoft/khulnasoft-realworld-app	A real-world example payment application. Uses GitHub Actions and CircleCI.
khulnasoft-gh-action-monorepo	Splits install and running tests commands, runs Khulnasoft from sub-folder
khulnasoft-examples	Shows separate install job from parallel test jobs
khulnasoft-gh-action-split-jobs	Shows a separate install job with the build step, and another job that runs the tests

Notes

Installation

This action installs local dependencies using lock files. Ensure that exactly one type of lock file is used for each project or working-directory from the following supported package managers:

Lock file	Package Manager	Installation command
`package-lock.json`	npm	`npm ci`
`pnpm-lock.yaml`	pnpm	`pnpm install --frozen-lockfile`
`yarn.lock`	Yarn Classic	`yarn --frozen-lockfile`

See section Yarn Modern for information about using Yarn version 2 and later.

Debugging

This action uses the debug module to output additional verbose logs. You can see these debug messages by setting the following environment variable:

DEBUG: @khulnasoft/github-action

You can set the environment variable using GitHub UI interface, or in the workflow file:

- name: Khulnasoft tests with debug logs
  uses: khulnasoft/github-action@v6
  env:
    DEBUG: '@khulnasoft/github-action'

See the example-debug.yml workflow file.

To collect more verbose GitHub Action logs you can set a GitHub secret or variable ACTIONS_STEP_DEBUG to true. This is useful to see detailed caching steps. See Enabling debug logging from GitHub Actions documentation for more information.

Logs from the test runner

To see the Khulnasoft debug logs add an environment variable to the action:

- name: Khulnasoft tests with debug logs
  uses: khulnasoft/github-action@v6
  env:
    DEBUG: 'khulnasoft:*'

Debugging waiting for URL to respond

If you have a problem with wait-on not working, you can check the src/ping.js logic from the local machine.

clone this repository to the local machine
install dependencies with npm install
start your server
from another terminal call the ping yourself to validate the server is responding:

$ node src/ping-cli.js <url>

For example

$ node src/ping-cli.js https://example.khulnasoft.com
pinging url https://example.khulnasoft.com for 30 seconds
::debug::pinging https://example.khulnasoft.com has finished ok

More information

Read our blog post Drastically Simplify Testing on CI with Khulnasoft GitHub Action
Read Test the Preview Vercel Deploys blog post
Creating actions docs

Extras

Manual trigger

If you add workflow_dispatch event to your workflow, you will be able to start the workflow by clicking a button on the GitHub page, see the Test External Site Using GitHub Actions video.

Outputs

This action sets a GitHub step output resultsUrl if the run was recorded on Khulnasoft Cloud using the action parameter setting record: true (see Record test results on Khulnasoft Cloud). Note that using a Custom test command with the command parameter overrides the record parameter and in this case no resultsUrl step output is saved.

This is an example of using the step output resultsUrl:

- name: Khulnasoft tests
  uses: khulnasoft/github-action@v6
  # let's give this action an ID so we can refer
  # to its output values later
  id: khulnasoft
  with:
    record: true
  env:
    KHULNASOFT_RECORD_KEY: ${{ secrets.RECORDING_KEY }}
- name: Print Khulnasoft Cloud URL
  if: always()
  run: |
    echo Khulnasoft finished with: ${{ steps.khulnasoft.outcome }}
    echo See results at ${{ steps.khulnasoft.outputs.resultsUrl }}

The GitHub step output dashboardUrl is deprecated. Khulnasoft Dashboard is now Khulnasoft Cloud.

Note: every GitHub workflow step can have outcome and conclusion properties. See the GitHub Contexts documentation section steps context. In particular, the outcome or conclusion value can be success, failure, cancelled, or skipped which you can use in any following steps.

Print Khulnasoft info

Sometimes you might want to print Khulnasoft and OS information, like the list of detected browsers. You can use the khulnasoft info command for this.

If you are NOT using the build command in your project, you can run the khulnasoft info command:

name: info
on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          build: npx khulnasoft info

If you are already using the build parameter, you can split the installation and the test steps and insert the khulnasoft info command in between:

name: info
on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Khulnasoft install
        uses: khulnasoft/github-action@v6
        with:
          # just perform install
          runTests: false
      - name: Khulnasoft info
        run: npx khulnasoft info
      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          # we have already installed all dependencies above
          install: false
          # rest of your parameters

Nightly tests

Sometimes you want to execute the workflow on a schedule. For example, to run Khulnasoft tests nightly, you can schedule the workflow using cron syntax:

name: example-cron
on:
  schedule:
    # runs tests every day at 4am
    - cron: '0 4 * * *'
jobs:
  nightly:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Khulnasoft nightly tests 🌃
        uses: khulnasoft/github-action@v6

Job summary title

By default, the action produces a job summary in the GitHub Actions log for each workflow step where github-action is used. Each job summary shows a Passing / Failing status, the test counts for Passed, Failed, Pending & Skipped, followed by the Duration of the run. The job summaries are grouped by job.

To specify a title for a Job Summary, use the parameter summary-title. If no title is specified, then the default "Khulnasoft Results" is used:

name: Summary titles
on: push
jobs:
  tests:
    runs-on: ubuntu-22.04
    steps:
      - uses: actions/checkout@v4
      - name: Khulnasoft headless tests
        uses: khulnasoft/github-action@v6
        with:
          summary-title: 'Headless'
      - name: Khulnasoft headed tests
        uses: khulnasoft/github-action@v6
        with:
          install: false
          headed: true
          summary-title: 'Headed'

The name of the GitHub Actions job is shown at the top of one or more job summaries from the same job. If multiple summaries belong to the same job, then giving them separate titles allows them to be uniquely identified.

See the example-chrome.yml workflow, with multiple calls to khulnasoft/github-action in one job, making use of the summary-title parameter. View the example-chrome.yml - actions log for an example of the resulting job summaries.

Suppress job summary

The default job summary can be suppressed by using the parameter publish-summary and setting its value to false.

name: Example no summary
on: push
jobs:
  khulnasoft-run:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Khulnasoft run
        uses: khulnasoft/github-action@v6
        with:
          publish-summary: false

Node.js

Support

Node.js is required to run this action. The recommended version v6 supports:

Node.js 18.x
Node.js 20.x and above

and is generally aligned with Node.js's release schedule.

Usage

github-action command-type options such as install-command, build, start and command are executed with the runner's version of Node.js. You can use GitHub's actions/setup-node to install an explicit Node.js version into the runner.

Khulnasoft itself runs with a fixed Node.js version specified by the runs.using parameter of action.yml. github-action@v6 uses node20.

Changelog

View the CHANGELOG document for an overview of version changes.

Compatibility

v6 is the current recommended version and uses node20
v5 uses the deprecated version node16
v4 is the minimum version required for Khulnasoft 10.x and later - also uses the deprecated version node16

Pay attention to any GitHub Actions deprecation warnings shown in logs which may recommend updating.

Contributing

Please see

actions khulnasoft

@actions/cache @actions/core @actions/exec @actions/io @octokit/core argument-vector debug find-yarn-workspace-root got hasha quote supports-color

0.0.0-development

2 years ago