ado-test-history v0.0.7

ado-test-history

This is a light utility to analyze history of failing tests from an Azure DevOps build.

Prerequisites:

• install node so that npx works in your shell
• install the Azure CLI and run az login in your shell

Usage:

npx ado-test-history --org <org> --project <project> --buildId <buildId>

Customization

This utility was designed against a particular ADO build/repo so the defaults may not generalize for others. In order of usefulness:

• --buildDefinitionBaselineID 123 allows you to run this utility against something like a PR build. The visualization shows historical test results from definition 123 (ideally something stable like CI)
• --buildReferenceNumberFilter X The ADO API filters records to a particular build definition, but includes all branches. Setting X here limits records to builds whose name includes X (master by default)
• --verbose will create JSON files in a log directory. This should work for most customization since you can parse the files yourself.

Suggested commit logic

The utility prints commits associated with builds that failed once after --passingBuildStableThreshold consecutive passed builds.

If buildDefinitionBaselineID is set, we suggest commits regardless, even if the latest baseline test is passing.

Otherwise, we only suggest commits if the latest baseline test is failing.