cypress-xray-junit-reporter v1.1.7

Writing XML report compatible with XRAY & JIRA

Enhances your Cypress test suite with the cypress-xray-junit-reporter a specialized custom reporter designed to seamlessly generating comprehensive XRay-compatible JUnit-style XML reports, complete with embedded screenshots on test failures, facilitating a thorough analysis of test execution.
This tailor-made reporter not only aligns with the best practices outlined in XRAY's guide on "Taking advantage of JUnit XML reports" but also leverages Cypress's "Custom reporter" capabilities.

XML cypress custom reporter based on Mocha to be compatible with:

• XRAY | Native Test Management for Jira
• CYPRESS V10+ | Testing framework

This plugin will also add support for two new cypress features:

• deleteVideoOnPassed (delete the videos of passed specs)
• betterRetries (logs cypress errors on retries)

See here for more information

Video reporter execution

https://github.com/alecmestroni/cypress-xray-junit-reporter/assets/62354989/4e8b1067-59bd-48ef-9f1a-183cfb049864

Identified Issue

Cypress presently lacks support for transmitting variables to the reporter when using the Electron browser.
This results in the impaired functionality of the "attachScreenshot" feature within this specific context. It is highly recommended to utilize alternative browsers for an optimal experience.

Install

$ npm install cypress-xray-junit-reporter --save-dev

or as a global module

$ npm install -g cypress-xray-junit-reporter

XRAY Mode

With this custom report will be easy for you to connect your tests with your JIRA test issue, creating test execution report compatible with XRAY.

1. Naming the output file

By default if a file test-report.xml already exists it will be overwritten.
The use of placeholders enables support of parallel execution of multiple test,cypress-xray-junit-reporter will write test results in separate files.
The mochaFile option can contain placeholders, e.g. ./path_to_your/test-results.[hash].xml.
In addition to [hash], these can also be used:

2. Cypress configuration

2.1 Inside cypress.config.js

This example shows how to install the plugin for e2e testing type. Read Cypress configuration docs for further info.

const { defineConfig } = require('cypress')
module.exports = defineConfig({
	deleteVideoOnPassed: true,
	betterRetries: true,
	reporter: 'cypress-xray-junit-reporter',
	reporterOptions: {
		mochaFile: './report/[suiteName].xml',
		useFullSuiteTitle: false,
		jenkinsMode: true,
		xrayMode: true, // if JiraKey are set correctly inside the test the XML report will contain the JiraKey value
		attachScreenshot: true, // if a test fails, the screenshot will be attached to the XML report and imported into xray
	},
	e2e: {
		setupNodeEvents(on, config) {
			require('cypress-xray-junit-reporter/plugin')(on, config, {}) // also needed
			return config
		},
	},
})

2.2 Inside cypress/support/e2e.js

At the top of your support file (usually cypress/support/e2e.js for e2e testing type):

import 'cypress-xray-junit-reporter/support'

3. Setting up your jiraKeys

cypress/e2e/myFirstTest.cy.js

describe('My First Test', () => {
	it('Does not do much!', { jiraKey: 'CALC-1234' }, () => {
		expect(true).to.equal(true)
	})
})

4. Run the test

npx cypress run

5. Enjoy the generated test-execution report

Report file generated at '<Cypress_project_root>/cypress/results'.

my-test-output-828a1c4885dc687b1a19e11e24b9437e.xml

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="Mocha Tests" time="0.1070" tests="1" failures="0">
  <testsuite name="Root Suite" timestamp="2023-01-27T13:51:23" tests="0" file="cypress\e2e\test.cy.js" time="0.0000" failures="0">
  </testsuite>
  <testsuite name="My First Test" timestamp="2023-01-27T13:51:23" tests="1" time="0.0700" failures="0">
    <testcase name="My First Test Does not do much!" time="0.0820" classname="Does not do much!">
      <properties>
        <property name="test_key" value="CALC-1234"/>
      </properties>
    </testcase>
  </testsuite>
</testsuites>

As you can see the property has been added and now could be read correctly by XRAY.

<properties>
  <property name="test_key" value="CALC-1234"/>
</properties>

Now just upload the report to XRAY and the card in Jira will be updated automatically

Other Options

Attach screenshot on failure

If you want to attach the screenshot of the failed testCase into the XML, you can use the attachScreenshot reporter option. The screenshot will be automatically converted into base64 and attached to the XML report.

<property name="testrun_evidence">
    <item name="image1.png">base64Here</item>
</property>

The testRun will include the screenshot as following:

Switch classname and name

If you want to switch classname and name of the generated testCase XML entries, you can use the testCaseSwitchClassnameAndName reporter option.

Here is an example of the XML output when using the testCaseSwitchClassnameAndName option:

You can also configure the testsuites.name attribute by setting reporterOptions.testsuitesTitle and the root suite's name attribute by setting reporterOptions.rootSuiteTitle.

Main reporterOptions list

Extra reporterOptions list (Advanced options)

Logged Results

Successful configuration

Description
All test cases are executed without any skips or pending status. Additionally, each test case is appropriately configured with the correct Jira key.
Cypress result:

────────────────────────────────────────────────────────────────────────────────────────────────────

  Running:  myFirstTest.cy.js                                                               (1 of 1)

  testSuite 1
    testSuite 2
      √ testCase 2.1
      √ testCase 2.2
    testSuite 3
      √ testCase 3.1
      √ testCase 3.2

 4 passing (475ms)

────────────────────────────────────────────────────────────────────────────────────────────────────

shortenLogMode disabled

====================================================================================================

  Cypress Xray Junit Reporter | Creating XML report
  -------------------------------------------------

    ⏳  XrayMode Enabled -> Retrieving suites information from Root Suite...

    〰 Founded one testsuite(s), keep scraping..
      〰 Analyzing testsuite: testSuite 1
      🔍 Looking for testsuite or testcase...

      〰 Founded two testsuite(s), keep scraping..
        〰 Analyzing testsuite: testSuite 2
        🔍 Looking for testsuite or testcase...
          〰 Properly analyzed testcase: testCase 2.1
          〰 Properly analyzed testcase: testCase 2.2
        ✔  Successfully analyzed two testcase(s)
        〰 End of testsuite: testSuite 2

        〰 Analyzing 3rd testsuite: testSuite 3
        🔍 Looking for testsuite or testcase...
          〰 Properly analyzed testcase: testCase 3.1
          〰 Properly analyzed testcase: testCase 3.2
        ✔  Successfully analyzed two testcase(s)
        〰 End of testsuite: testSuite 3

      〰 End of testsuite: testSuite 1

    ------------------------------------
    All suites has been parsed correctly!

====================================================================================================

shortenLogMode enabled

====================================================================================================

  Cypress Xray Junit Reporter | Creating XML report
  -------------------------------------------------

    ⏳  XrayMode Enabled -> Retrieving suites information from Root Suite...

    ------------------------------------
    All suites has been parsed correctly!

====================================================================================================

Missing jiraKeys

Description
Jira keys are missing in testCase 1.2 & testCase 1.3.
Cypress result:

────────────────────────────────────────────────────────────────────────────────────────────────────

  Running:  myFirstTest.cy.js                                                               (1 of 1)

  testSuite 1
    √ testCase 1.1
    √ testCase 1.2
    √ testCase 1.3

 3 passing (185ms)

────────────────────────────────────────────────────────────────────────────────────────────────────

shortenLogMode disabled

====================================================================================================

  Cypress Xray Junit Reporter | Creating XML report
  -------------------------------------------------

    ⏳  XrayMode Enabled -> Retrieving suites information from Root Suite...

    〰 Founded one testsuite(s), keep scraping..
      〰 Analyzing testsuite: testSuite 1
      🔍 Looking for testsuite or testcase...
        〰 Properly analyzed testcase: testCase 1.1
        〰 Properly analyzed testcase: testCase 2.1
        ⚠️ Missing jira key in testcase: testCase 1.3
        〰 Skipping 3rd testcase: testCase 1.3
      ✔  Successfully analyzed three testcase(s)
      ❗ Missing jira key in at least one testcase
      〰 End of testsuite: testSuite 1

  ------------------------------------
  All suites has been parsed correctly!

====================================================================================================

shortenLogMode enabled

====================================================================================================

  Cypress Xray Junit Reporter | Creating XML report
  -------------------------------------------------

    ⏳ XrayMode Enabled -> Retrieving suites information from Root Suite...
    ⚠️ Missing jira key in testcase: testCase 1.3
    ‼ Missing jira key in at least one testcase
    ‼ Skipping testcases:
    - testCase 1.3

  ------------------------------------
  All suites have been parsed correctly!

====================================================================================================

Skipped or Pending tests

Description Skipping or Pending tests will be skipped
Cypress result:

────────────────────────────────────────────────────────────────────────────────────────────────────

  Running:  myFirstTest.cy.js                                                               (1 of 1)

 testSuite 1
      √ testCase 1.1
      - testCase 1.2
      - testCase 1.3

  1 passing (398ms)
  2 pending

────────────────────────────────────────────────────────────────────────────────────────────────────

shortenLogMode disabled

====================================================================================================

  Cypress Xray Junit Reporter | Creating XML report
  -------------------------------------------------

    ⏳  XrayMode Enabled -> Retrieving suites information from Root Suite...

    〰 Founded one testsuite(s), keep scraping..
      〰 Analyzing testsuite: testSuite 1
      🔍 Looking for testsuite or testcase...
        〰 Properly analyzed testcase: testCase 1.1
        〰 Skipping testcase: testCase 1.2
        〰 Skipping testcase: testCase 1.3
      ✔  Successfully analyzed three testcase(s)
      〰 End of testsuite: testSuite 1

  ------------------------------------
  All suites has been parsed correctly!

====================================================================================================

shortenLogMode enabled

====================================================================================================

  Cypress Xray Junit Reporter | Creating XML report
  -------------------------------------------------

    ⏳  XrayMode Enabled -> Retrieving suites information from Root Suite...

    ‼ Skipping testcases:
    - testCase 1.2,
    - testCase 1.3

  ------------------------------------
  All suites has been parsed correctly!

====================================================================================================

Module dependencies error

Description
Try reinstall the latest version of the library

────────────────────────────────────────────────────────────────────────────────────────────────────

  Running:  myFirstTest.cy.js                                                               (1 of 1)
"cypress-xray-junit-reporter" reporter not found
Reporter not found! cypress-xray-junit-reporter

  testSuite 1
    √ testCase 1.1

 1 passing (142ms)

====================================================================================================

Extra Features

Set them as other cypress options inside the cypress.config.js:

const { defineConfig } = require('cypress')
module.exports = defineConfig({
	deleteVideoOnPassed: true,
	betterRetries: true,
})

deleteVideoOnPass

Deletes the videos of passed specs

====================================================================================================

Test-Run "myFirstTest": SUCCESS!
Deleting video output

====================================================================================================

betterRetries

Cypress doesn't automatically logs retries errors but log only the last one.
In some cases you need to know the error on each attempt because it can change.
Before betterRetries:

────────────────────────────────────────────────────────────────────────────────────────────────────

  Running:  myFirstTest.cy.js                                                               (1 of 1)

  testSuite 1
    1) testCase 1.1

 0 passing (1s)
  1 failing

  1) testSuite 1
       testCase 1.1:
     AssertionError: expected true to equal false
      at Context.eval (webpack://plain-iqpfe-cypresstest/./cypress/e2e/tests/myFirstTest.cy.js:6:18)


====================================================================================================

After betterRetries:

────────────────────────────────────────────────────────────────────────────────────────────────────

  Running:  myFirstTest.cy.js                                                               (1 of 1)

  testSuite 1
    (Attempt 1 of 3) testCase 1.1
    AssertionError: expected true to equal false
    (Attempt 2 of 3) testCase 1.1
    AssertionError: expected true to equal false
    1) testCase 1.1
    (Attempt 3 of 3) testCase 1.1

 0 passing (1s)
  1 failing

  1) testSuite 1
       testCase 1.1:
     AssertionError: expected true to equal false
      at Context.eval (webpack://plain-iqpfe-cypresstest/./cypress/e2e/tests/myFirstTest.cy.js:6:18)


====================================================================================================

THE JOB IS DONE!

Happy testing to everyone!

ALEC-JS