Upssert NPM | npm.io

Upssert

Upssert is an extremely simple REST API assertion framework which makes it easier to monitor and test your API. Without writing any code, test your API against its responsiveness and any combination of parameters.

Principles

Zero configuration setup
Human-readable test suite definitions
Repeatable across environments
Assert anything
Stay easy

Installation

npm install upssert

Usage

$ upssert --url http://httpbin.org/get

Executing 1 test suites (2 assertions)…

Ping
  ✓ http://httpbin.org/get

1 passing (204.000ms)

By default Upssert will assert that a URL's response code is within the 200 or 300 range, i.e.

$ node ./dist/bin/index.js --url https://httpbin.org/status/404

Executing 1 test suites (2 assertions)…

Ping
  ✖ https://httpbin.org/status/404

1) https://httpbin.org/status/404
Error: expected 404 to be below 400 (statusCode)

0 passing (418.000ms)
1 failing

Assertions

Under the hood, Upssert uses Chai Assertion Library's assert API for its assertions. All the assertions exposed by Chai are available in Upssert with the exception of:

assert(expression, message)
throws
doesNotThrow
changes
doesNotChange
increases
doesNotIncrease
decreases
doesNotDecrease

The following will be supported in the future

operater
closeTo
approximately

CLI

Options

url Supply Upssert with a URL to run a ping assertion against
reporter Tell Upssert which reporter to use. Options are tap and console. Defaults to console

If neither a glob pattern or --url flag is supplied, Upssert will look in test/api/**/*.json for test suites. Alternatively, Upssert can be configured within package.json with the following options:

{
  ...
  "upssert": {
    "globOpts": {},
    "testDir": "path/to/your/tests/**/*.json"
  }
}

globOpts can be configured as defined in https://github.com/isaacs/node-glob

Running Test Suites

Test suites are defined as JSON files

$ cat << EOF > ping.json
{
  "name": "Ping httpbin.org",
  "tests": [
    {
      "name": "Successfully pings httpbin.org",
      "request": {
        "url": "https://httpbin.org",
        "method": "GET"
      }
    }
  ]
}
EOF

$ upssert ping.json

A more complex example

$ cat << EOF > workflow.json
{
  "name": "Test against httpbin",
  "tests": [
    {
      "name": "/get",
      "request": {
        "url": "https://httpbin.org/get?foo=bar",
        "method": "GET"
      },
      "response": {
        "content-type": "application/json",
        "status-code": {
          "equal": 200
        },
        "timing.dns-resolution": {
          "is-below": 100
        },
        "timing.total": {
          "is-below": 500
        },
        "body.args.foo": "bar"
      }
    }, {
      "name": "/post formdata",
      "requires": ["test1"],
      "request": {
        "url": "https://httpbin.org/post",
        "method": "POST",
        "form": [
          {
            "key": "out",
            "value": "{{test1.body.args.foo}}"
          }
        ]
      },
      "response": {
        "content-type": "application/json",
        "status-code": {
          "equal": 200
        },
        "timing.dns-resolution": {
          "is-below": 100
        },
        "timing.total": {
          "is-below": 500
        },
        "body.form.out": "bar"
      }
    }, {
      "name": "/post raw data",
      "requires": ["test1", "test2"],
      "request": {
        "url": "https://httpbin.org/post",
        "method": "POST",
        "headers": {
          "Content-Type": "application/x-www-form-urlencoded"
        },
        "data": "test1-data={{test1.body.args.foo}}&test2-data={{test2.body.form.out}}"
      },
      "response": {
        "content-type": "application/json",
        "status-code": {
          "equal": 200
        },
        "timing.dns-resolution": {
          "is-below": 100
        },
        "timing.total": {
          "is-below": 500
        },
        "body.form[test1-data]": "bar",
        "body.form[test2-data]": "bar"
      }
    }
  ]
}
EOF

$ upssert workflow.json

Programmatically

import Upssert from 'upssert';
const ping = {
  name: 'Ping httpbin.org',
  tests: [
    {
      name: 'Successfully pings httpbin.org',
      request: {
        url: 'https://httpbin.org',
        method: 'GET',
      },
    },
  ],
}

const upssert = new Upssert(ping);
upssert.execute();

Reporters

Currently, there are only two reporters available

ConsoleReporter (default)

Similar to Mocha Spec reporter.

TapReporter

Outputs test reports with the TAP specification

import Upssert, { TapReporter } from 'upssert';
const reporter = new TapReporter();
const upssert = new Upssert('https://httpbin.org/get', reporter);
upssert.execute();

1..1
ok 1 https://httpbin.org/get
# tests 1
# pass 1
# fail 0
# assertions 2

Lifecycle

Upssert uses the EventEmitter to emit events at runtime. The bundled reporters report on events as they occur instead of collecting all output first and then writing out the formatted report.

The following is an exhaustive list of events which are emitted

suite.assertion.count (count) emits count assertions for each test
suite.test.count (count) emits count tests for each suite
suite.count (count) emits count for each suite, which is usually 1
start emitted when the test runner starts
suite.start emitted when a suite starts
suite.fail emitted if a suite fails and cannot continue. Generally for invalid test suite configuration.
suite.test.start emitted when a test starts
suite.test.pass emitted if none of the tests assertions fail
suite.test.fail emitted if any of the tests assertions fail
suite.test.end emitted when a test ends
fail emitted for every failure (suite.test.fail, suite.fail). Useful for marking a test run as failed.
suite.end emitted when a suite ends
end emitted when the test runner stops

Custom Reporters

User-defined reporters can be passed to Upssert through the constructor.

A custom implementation only requires that it expose a setEventEmitter(emitter) method which Upssert can pass an event emitter into.

class CustomReporter {
  setEventEmitter(emitter){
    emitter.on('start', () => console.log('Started'));
    emitter.on('end', () => console.log('Ended'));
  }
}

const upssert = new Upssert('https://httpbin.org/get', new CustomReporter());
upssert.execute();

Inspired by the simplicity of Ansible