Reftest-runner NPM

Reftest-runner

A visual testing tool for Browser(HTML).

overview

reftest-runner is similar testing tools to Layout Engine Visual Tests (reftest).

Layout Engine Visual Tests (reftest) is adopted major browser Firefox, Chrome, Safari etc.

reftest-runner concept is the same, but use it with any browser that supported WebDriver API.

A reftest is a test that compares the visual output of one file (the test case) with the output of one or more other files (the references). The test and the reference must be carefully written so that when the test passes they have identical rendering, but different rendering when the test fails.

Installation

npm install reftest-runner

Require Node.js v4 >=

Feature

Compare the visual output of HTMLs.
- for testing Canvas, HTML, CSS etc..
Compare the visual output of browsers.
- e.g.) Firefox vs Chrome.
Output diff image
- mismatch the visual, then output diff image of these.
Output test result as TAP format.
reftest.list support
- azu/reftest-list-parser
WebDriver API support
- This tools running on Firefox/Chrome/IE/PhantomJS and more?

Usage

Command line

npm install reftest-runner -g

If you want to compare path/to/fileA.html and path/to/fileB.html using phantomjs.

$ reftest-runner --browser "phantomjs" --targetA path/to/fileA.html --targetB path/to/fileB.html

also use reftest.lit file for test.

$ reftest-runner --list path/to/reftest.list

You can view about reftest.list format on azu/reftest-list-parser.

Command line help

reftest-runner [options]

Options:
  -h, --help                Show help
  -l, --list path::String   Use reftest list from this file
  --targetA path::String    Use a specific test html file
  --targetB path::String    Use a specific test html file
  -v, --version             Outputs the version number
  -b, --browser String      Specify Browser - default: phantomjs
  --compareOperator String  Specify compareOperator. == OR != - default: ==
  --useExternalServer       Use external server url

Command line interface is limited.

if you want to flexibility, please suggestion to issue or use it as node modules.

Node modules

Please see the Examples. :bulb:

reftest-runner export Engine and Runner.

Engine is wrapper of Runner for treating multiple files and using local server...
Runner is core module.

module.exports = {
    Engine: require("./reftest-engine"),
    Runner: require("./reftest-runner")
};

Example: Programmatic run test with reftest.list

var path = require("path");
var Promise = require("bluebird");
var ReftestEngine = require("reftest-runner").Engine;
// options
// see https://github.com/azu/reftest-runner/blob/master/typings/reftest-runner/reftest-runner.d.ts
//     https://github.com/azu/reftest-runner/blob/master/src/options/default-options.js
var testEngine = new ReftestEngine({
    server: {
        port: 8989
    },
    rootDir: __dirname
});
function allPassed(resultList) {
    return resultList.every(function (result) {
        return result.passed;
    });
}

// run test with reftest.list
function reftestWithList(reftestListPath) {
    var list = testEngine.getTargetListFromFile(reftestListPath);
    return testEngine.runTests(list).then(function (resultList) {
        var formatter = testEngine.getReporter();
        var output = formatter(resultList);
        console.log(output);
        if (!allPassed(resultList)) {
            return Promise.reject(new Error("FAIL"));
        }
    });
}

var reftestListPath = path.join(__dirname, "reftest.list");
reftestWithList(reftestListPath).catch(function (error) {
    console.error(error.message);
    console.error(error.stack);
});