0.0.1 • Published 4 years ago
flaky-tracker v0.0.1
flaky-tracker
Project that scans and extract flaky tests from Buildkite pipelines
Installation
- Create an S3 bucket
Upload the Buildkite token to s3 Bucket:
- The S3 object key name should be of the form
<Organization Name>/config.json
The content should be
{ "buildkite_api_key": "<Buildkite API Key>" }
- The S3 object key name should be of the form
Install library:
yarn add flaky-tracker
Usage
Call the
extractFlakySpec
. For Example:import extractFlakySpec, { aggregateArtifactStrategy, flakyJobsOutputStrategy, } from "flaky-tracker"; await extractFlakySpec( "<S3 bucket Name>", "<Organization Name>", "<Pipeline Name>", "<Pipeline Slug>", flakyJobsOutputStrategy, );
The above function will scan the builds from the last 2 days and upload the flaky specs to the S3 Bucket. You can use Quicksight to create graphs/tables using the S3 Bucket.
Supported strategies
flakyJobsOutputStrategy
This strategy scans the rspec output of jobs that passed on retry. It works if you use the default rspec output, and do not retry inside the same job
aggregateArtifactStrategy
This strategy works if you aggregate all failures in an artifacts
tmp/non-deterministic-and-failures.json
, even when the build passes
0.0.1
4 years ago