0.0.10 • Published 5 months ago

newman-reporter-ctrf-json v0.0.10

Weekly downloads
-
License
MIT
Repository
github
Last release
5 months ago

Newman Postman JSON test results report

Save Newman Postman test results as a JSON file

A Postman newman JSON test reporter to create test reports that follow the CTRF standard.

Common Test Report Format ensures the generation of uniform JSON test reports, independent of programming languages or test framework in use.

Features

  • Generate JSON test reports that are CTRF compliant
  • Straightforward integration with newman
{
  "results": {
    "tool": {
      "name": "newman"
    },
    "summary": {
      "tests": 1,
      "passed": 1,
      "failed": 0,
      "pending": 0,
      "skipped": 0,
      "other": 0,
      "start": 1706828654274,
      "stop": 1706828655782
    },
    "tests": [
      {
        "name": "ctrf should generate the same report with any tool",
        "status": "passed",
        "duration": 100
      }
    ],
    "environment": {
      "appName": "MyApp",
      "buildName": "MyBuild",
      "buildNumber": "1"
    }
  }
}

What is CTRF?

CTRF is a universal JSON test report schema that addresses the lack of a standardized format for JSON test reports.

Consistency Across Tools: Different testing tools and frameworks often produce reports in varied formats. CTRF ensures a uniform structure, making it easier to understand and compare reports, regardless of the testing tool used.

Language and Framework Agnostic: It provides a universal reporting schema that works seamlessly with any programming language and testing framework.

Facilitates Better Analysis: With a standardized format, programatically analyzing test outcomes across multiple platforms becomes more straightforward.

Installation

npm install newman-reporter-ctrf-json

Run your tests with the reporter argument via the cli:

newman run ./postman_collection.json -r cli,ctrf-json

or programmatically:

const newman = require('newman') // require newman in your project

// call newman.run to pass `options` object and wait for callback
newman.run(
  {
    collection: require('./sample-collection.json'),
    reporters: ['cli', 'ctrf-json'],
  },
  function (err) {
    if (err) {
      throw err
    }
    console.log('collection run complete!')
  }
)

You'll find a JSON file named ctrf-report.json in the ctrf directory.

Reporter Options

The reporter supports several configuration options, you can pass these via the command line:

newman run ./postman_collection.json -r cli,ctrf-json \
--reporter-ctrf-json-output-file custom-name.json \
--reporter-ctrf-json-output-dir custom-directory \
--reporter-ctrf-json-test-type api \
--reporter-ctrf-json-minimal false \
--reporter-ctrf-json-app-name MyApp \
--reporter-ctrf-json-app-version 1.0.0 \
--reporter-ctrf-json-os-platform linux \
--reporter-ctrf-json-os-release 18.04 \
--reporter-ctrf-json-os-version 5.4.0 \
--reporter-ctrf-json-build-name MyApp \
--reporter-ctrf-json-build-number 100 \
--reporter-ctrf-json-build-url https://ctrf.io \
--reporter-ctrf-json-repository-name ctrf \
--reporter-ctrf-json-repository-url https://github.com/ctrf-io/newman-reporter-ctrf-json \
--reporter-ctrf-json-branch-name main \
--reporter-ctrf-json-test-environment staging

or programmatically:

const newman = require('newman') // require newman in your project

// call newman.run to pass `options` object and wait for callback
newman.run(
  {
    collection: require('./sample-collection.json'),
    reporters: ['cli', 'ctrf-json'],
    reporter: {
      'ctrf-json': {
        outputFile: 'api_report_ctrf.json',
        outputDir: 'api_reports',
        minimal: true,
        testType: 'api',
        appName: 'MyApp',
        appVersion: '1.0.0',
        osPlatform: 'linux',
        osRelease: '18.04',
        osVersion: '5.4.0',
        buildName: 'MyApp',
        buildNumber: '100',
        buildUrl: 'https://ctrf.io',
        repositoryName: 'ctrf',
        repositoryUrl: 'https://github.com/ctrf-io/newman-reporter-ctrf-json',
        branchName: 'main',
        testEnvironment: 'staging',
      },
    },
  },
  function (err) {
    if (err) {
      throw err
    }
    console.log('collection run complete!')
  }
)

Test Object Properties

The test object in the report includes the following CTRF properties:

NameTypeRequiredDetails
nameStringRequiredThe name of the test.
statusStringRequiredThe outcome of the test. One of: passed, failed, skipped, pending, other.
durationNumberRequiredThe time taken for the test execution, in milliseconds.
messageStringOptionalThe failure message if the test failed.
traceStringOptionalThe stack trace captured if the test failed.
suiteStringOptionalThe suite or group to which the test belongs.
typeStringOptionalThe type of test (e.g., api, contract).

Support Us

If you find this project useful, consider giving it a GitHub star ⭐ It means a lot to us.

0.0.10

5 months ago

0.0.9

7 months ago

0.0.8

7 months ago

0.0.5

11 months ago

0.0.4

11 months ago

0.0.7

11 months ago

0.0.6

11 months ago

0.0.3

1 year ago

0.0.2

1 year ago

0.0.1

1 year ago