page-load-tests v3.0.0

Page Load Tests

Page Load Tests is designed to automate tests requiring a web page loaded in a browser window (e.g. 404 error loading an external resource, page renders with horizontal scrollbars). The specific tests are outlined below, but are all general tests that do not include any page-specific logic.

Installation

Install Page Load Tests globally (as shown below), or locally, via npm.

npm install -g page-load-tests

Usage

Page Load Tests runs as a command line tool and is executed as follows:

Installed globally:
> page-load-tests

Installed locally:
> npx page-load-tests

Page Load Tests requires a configuration file named .pltconfig.json in the project root. This file provides the URLs to be tested and options to configure the test. Details on the available tests and the configuration file format are provided below.

Test Cases

The tests use Puppeteer to launch a headless Chrome browser. The URLs defined in the configuration file are each loaded once, and after page load the applicable tests are executed.

Horizontal Scrollbar Test

The horizontal scrollbar test fails if the rendered page has a horizontal scrollbar. If a specific browser viewport size is desired for this test, that can be configured in the puppeteerLaunchOptions.

Console Output Test

The console output test fails if any output is written to the browser console. An array is included in the report with all entries, as shown below:

Received array:
[
    {
        "_args": [],
        "_location": {
            "lineNumber": undefined,
            "url": "https://this.url.does.not.exist/file.js"
        },
        "_text": "Failed to load resource: net::ERR_NAME_NOT_RESOLVED",
        "_type": "error"
    }
]

Console Error Test

The console error test fails if any error is written to the browser console, but is otherwise the same as the console output test. This separation allows for testing for console errors, but allowing any other console output.

Rendered HTML Test

The rendered HTML test is intended for cases where content is dynamically created prior to page load (i.e. the load event firing). The rendered HTML is returned and checked with HTML Hint and the test fails if any issues are found. An array is included in the report with all HTML Hint issues, as shown below:

Received array:
[
    {
        "col": 9,
        "evidence": "    <div id=\"div1\"></div>",
        "line": 6,
        "message": "The id value [ div1 ] must be unique.",
        "raw": " id=\"div1\"",
        "rule": {
            "description": "The value of id attributes must be unique.",
            "id": "id-unique",
            "link": "https://github.com/thedaviddias/HTMLHint/wiki/id-unique"
        },
        "type": "error"
    }
]

Page Load Time Test

The page load time test fails if the page load time (from start through the load event) exceeds the defined threshold in the configuration file (or the default of 2 seconds). The actual load time is included in the report. Tests will time out at twice the page load time threshold.

Reports

Page Load Test results are displayed in the console and saved in two reports in the project root directory:

• A JSON report named page-load-tests-results.json
• An HTML report named page-load-tests-results.html

Both reports contain:

• The execution time of the tests
• A summary of the total tests, passed tests, and failed tests
• The detailed test results, including the URL tested, list of tests performed on that URL with results, and, if applicable, any relevant data associated with the test failure (e.g. the console errors if the console error test fails).

Complete reports for the example case in this project (the tests as specified in the project .pltconfig.json file) can be found at the links above.

Configuration

Page Load Tests looks for a configuration file named .pltconfig.json in the project root. If the file is not found, is not valid JSON, or does not contain any URLs to check the job will fail.

Below is an example .pltconfig.json file, which is broken into three major sections:

• puppeteerLaunchOptions: An optional set of options to pass to Puppeteer on launch. There are no default options. The complete list of available options can be found at https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#puppeteerlaunchoptions.
• settings: These are settings to enable/disable tests or update test settings that are applied to all tests, overriding the defaults. All available settings with the default values are shown below.
• urls: An array of URLs to be tested, which must contain at least one value. Each array entry can either be a URL string, or an object that contains a url string and an optional settings object. This object can contain any of the settings values identified above and will override that setting for testing that URL. The url string can be either an actual URL or a local file, as shown in the example below.

{
    "puppeteerLaunchOptions": {
        "args": [ "--no-sandbox" ]
    },
    "settings": {
        "pageLoadTimeThreshold": 2,
        "horizontalScrollbarTest": true,
        "consoleOutputTest": true,
        "consoleErrorTest": true,
        "renderedHtmlTest": true,
        "pageLoadTimeTest": true
    },
    "urls": [
        "https://gitlab.com/gitlab-ci-utils/page-load-tests/",
        {
            "url": "./tests/test-cases/consoleLog.html",
            "settings": {
                "consoleOutputTest": false
            }
        }
    ]
}

Docker Images

Provided with the Page Load Test project are Docker images configured to run the tests. All available image tags can be found in the gitlab-ci-utils/page-load-tests repository at https://gitlab.com/gitlab-ci-utils/page-load-tests/container_registry. Details on each release can be found on the Releases page.

Note: Any images in the gitlab-ci-utils/page-load-tests/tmp repository are temporary images used during the build process and may be deleted at any point.

GitLab CI Configuration

The following is an example job from a .gitlab-ci.yml file to use this image to run Page Load Tests against another project in GitLab CI:

page_load_tests:
  image: registry.gitlab.com/gitlab-ci-utils/page-load-tests:latest
  stage: test
  script:
    - page-load-tests
  artifacts:
    when: always
    paths:
      - page-load-tests-results.html
      - page-load-tests-results.json