Bulk-release NPM

zx-bulk-release

zx-based alternative for multi-semantic-release

Key features

Conventional commits trigger semantic releases.
Automated cross-pkg version bumping.
Predictable toposort-driven flow.
No default branch blocking (no release commits).
Pkg changelogs go to changelog branch (configurable).
Docs are published to gh-pages branch (configurable).
No extra builds. The required deps are fetched from the pkg registry (npmFetch config opt).

Roadmap

Store release metrics to meta.
~~Self-repair. Restore broken/missing metadata from external registries (npm, pypi, m2)~~. Tags should be the only source of truth
Multistack. Add support for java/kt/py.
Semaphore. Let several release agents to serve the monorepo at the same time.

Requirements

macOS / linux
Node.js >= 16.0.0
npm >=7 / yarn >= 3
~~wget~~
tar
git

Usage

Install

yarn add zx-bulk-release

CLI

GH_TOKEN=ghtoken GH_USER=username NPM_TOKEN=npmtoken npx zx-bulk-release [opts]

Flag	Description	Default
`--ignore`	Packages to ignore: `a, b`
`--include-private`	Include `private` packages	`false`
`--concurrency`	`build/publish` threads limit	`os.cpus.length`
`--no-build`	Skip `buildCmd` invoke
`--no-test`	Disable `testCmd` run
`--no-npm-fetch`	Disable npm artifacts fetching
`--only-workspace-deps`	Recognize only `workspace:` deps as graph edges
`--dry-run` / `--no-publish`	Disable any publish logic
`--report`	Persist release state to file
`--snapshot`	Disable any publishing steps except of `npm` and `publishCmd` (if defined), then push packages to the `snapshot` channel
`--debug`	Enable zx verbose mode
`--version` / `-v`	Print own version

JS API

import { run } from 'zx-bulk-release'

const cwd = '/foo/bar'
const env = {GH_TOKEN: 'foo', NPM_TOKEN: 'bar'}
const flags = {dryRun: true}

await run({
  cwd,    // Defaults to process.cwd()
  flags,  // Defaults to process.env
  env     // Defaults to minimist-parsed `process.argv.slice(2)`
})

Config

cosmiconfig

Any cosmiconfig compliant format: .releaserc, .release.json, .release.yaml, etc in the package root or in the repo root dir.

{
  "cmd": "yarn && yarn build && yarn test",
  "npmFetch": true,
  "changelog": "changelog",
  "ghPages": "gh-pages"
}

env vars

export const parseEnv = (env = process.env) => {
  const {GH_USER, GH_USERNAME, GITHUB_USER, GITHUB_USERNAME, GH_TOKEN, GITHUB_TOKEN, NPM_TOKEN, NPM_REGISTRY, NPMRC, NPM_USERCONFIG, NPM_CONFIG_USERCONFIG, NPM_PROVENANCE, GIT_COMMITTER_NAME, GIT_COMMITTER_EMAIL} = env

  return {
    ghUser:             GH_USER || GH_USERNAME || GITHUB_USER || GITHUB_USERNAME,
    ghToken:            GH_TOKEN || GITHUB_TOKEN,
    npmToken:           NPM_TOKEN,
    // npmConfig suppresses npmToken
    npmConfig:          NPMRC || NPM_USERCONFIG || NPM_CONFIG_USERCONFIG,
    npmRegistry:        NPM_REGISTRY || 'https://registry.npmjs.org',
    npmProvenance:      NPM_PROVENANCE,
    gitCommitterName:   GIT_COMMITTER_NAME || 'Semrel Extra Bot',
    gitCommitterEmail:  GIT_COMMITTER_EMAIL || 'semrel-extra-bot@hotmail.com',
  }
}

Demo

Implementation notes

Flow

try {
  const {packages, queue, root} = await topo({cwd, flags})
  console.log('queue:', queue)

  for (let name of queue) {
    const pkg = packages[name]

    await analyze(pkg, packages, root)

    if (pkg.changes.length === 0) continue

    await build(pkg, packages)

    if (flags.dryRun) continue

    await publish(pkg)
  }
} catch (e) {
  console.error(e)
  throw e
}

`topo`

Toposort is used to resolve the pkg release queue. By default, it omits the packages marked as private. You can override this by setting the --include-private flag.

`analyze`

Determines pkg changes, release type, next version etc.

export const analyze = async (pkg, packages, root) => {
  pkg.config = await getPkgConfig(pkg.absPath, root.absPath)
  pkg.latest = await getLatest(pkg)

  const semanticChanges = await getSemanticChanges(pkg.absPath, pkg.latest.tag?.ref)
  const depsChanges = await updateDeps(pkg, packages)
  const changes = [...semanticChanges, ...depsChanges]

  pkg.changes = changes
  pkg.version = resolvePkgVersion(changes, pkg.latest.tag?.version || pkg.manifest.version)
  pkg.manifest.version = pkg.version

  console.log(`[${pkg.name}] semantic changes`, changes)
}

Set config.releaseRules to override the default rules preset:

[
  {group: 'Features', releaseType: 'minor', prefixes: ['feat']},
  {group: 'Fixes & improvements', releaseType: 'patch', prefixes: ['fix', 'perf', 'refactor', 'docs', 'patch']},
  {group: 'BREAKING CHANGES', releaseType: 'major', keywords: ['BREAKING CHANGE', 'BREAKING CHANGES']},
]

`build`

Applies config.cmd to build pkg assets: bundles, docs, etc.

export const build = async (pkg, packages) => {
  // ...
  if (!pkg.fetched && config.cmd) {
    console.log(`[${pkg.name}] run cmd '${config.cmd}'`)
    await $.o({cwd: pkg.absPath, quote: v => v})`${config.cmd}`
  }
  // ...
}

`publish`

Publish the pkg to git, npm, gh-pages, gh-release, etc.

export const publish = async (pkg) => {
  await fs.writeJson(pkg.manifestPath, pkg.manifest, {spaces: 2})
  await pushTag(pkg)
  await pushMeta(pkg)
  await pushChangelog(pkg)
  await npmPublish(pkg)
  await ghRelease(pkg)
  await ghPages(pkg)
}

tagFormat	Example
f0	2022.6.22-qiwi.pijma-native.v1.0.0-beta.0+foo.bar-f0
f1	2022.6.13-examplecom.v1.0.0.ZXhhbXBsZS5jb20-f1
lerna	@qiwi/pijma-ssr@1.1.12
pure	1.2.3-my.package

Report

Release process state is reported to the console and to a file if --report flag is set to /some/path/release-report.json, for example.

{
  status: 'success',            // 'sucess' | 'failure' | 'pending'
  error: null,                  // null or Error
  queue: ['a', 'b', 'c', 'd']   // release queue
  packages: [{
    name: 'a',
    version: '1.1.0',
    path: '/pkg/abs/path',
    relPath: 'pkg/rel/path',
    config: {                   // pkg config
      changelog: 'changelog',
      npmFetch: true
    },                 
    changes: [{                 // semantic changes
      group: 'Features',
      releaseType: 'minor',
      change: 'feat: add feat',
      subj: 'feat: add feat',
      body: '',
      short: '792512c',
      hash: '792512cccd69c6345d9d32d3d73e2591ea1776b5'
    }],                  
    tag: {
      version: 'v1.1.0',
      name: 'a',
      ref: '2022.6.22-a.v1.1.0-f0'
    },
    releaseType: 'minor',       // 'major' | 'minor' | 'patch'
    prevVersion: '1.0.0'        // previous version or null
  }, {
    name: 'b',
    // ...
  }],
  events: [
    {msg: ['zx-bulk-release'], scope:'~', date: 1665839585488, level: 'info'},
    {msg: ['queue:',['a','b']], scope:'~', date: 1665839585493, level: 'info'},
    {msg: ["run buildCmd 'yarn && yarn build && yarn test'"], scope: 'a', date: 1665839585719, level:'info'},
    // ...
  ]
}

Output

Compact and clear logs

Run npm_config_yes=true npx zx-bulk-release
zx-bulk-release
[@semrel-extra/zxbr-test-a] semantic changes [
  {
    group: 'Fixes & improvements',
    releaseType: 'patch',
    change: 'fix(a): random',
    subj: 'fix(a): random',
    body: '',
    short: '6ff25bd',
    hash: '6ff25bd421755b929ef2b58f35c727670fd93849'
  }
]
[@semrel-extra/zxbr-test-a] run cmd 'yarn && yarn build && yarn test'
[@semrel-extra/zxbr-test-a] push release tag 2022.6.27-semrel-extra.zxbr-test-a.1.8.1-f0
[@semrel-extra/zxbr-test-a] push artifact to branch 'meta'
[@semrel-extra/zxbr-test-a] push changelog
[@semrel-extra/zxbr-test-a] publish npm package @semrel-extra/zxbr-test-a 1.8.1 to https://registry.npmjs.org
[@semrel-extra/zxbr-test-a] create gh release
[@semrel-extra/zxbr-test-b] semantic changes [
  {
    group: 'Dependencies',
    releaseType: 'patch',
    change: 'perf',
    subj: 'perf: @semrel-extra/zxbr-test-a updated to 1.8.1'
  }
]
[@semrel-extra/zxbr-test-b] run cmd 'yarn && yarn build && yarn test'
[@semrel-extra/zxbr-test-b] push release tag 2022.6.27-semrel-extra.zxbr-test-b.1.3.5-f0
[@semrel-extra/zxbr-test-b] push artifact to branch 'meta'
[@semrel-extra/zxbr-test-b] push changelog
[@semrel-extra/zxbr-test-b] publish npm package @semrel-extra/zxbr-test-b 1.3.5 to https://registry.npmjs.org
[@semrel-extra/zxbr-test-b] create gh release
[@semrel-extra/zxbr-test-d] semantic changes [