Buildverse NPM

buildverse

The simpler monorepo toolkit for npm and friends

buildverse provides a Goldilocks amount of sweetness for making npm-compatible package managers into a first-class build tools command-line development environments for projects organised as a heirarchical collection of sub-projects/packages.

Note: This version has breaking changes over 1.x.x & 2.x.x versions. Please see the updated tutorial below.

All feedback kindly appreciated -- please refer to support.

About

buildverse is a suite of command-line tools for developing projects that are organised as a heirarchical collection of subprojects and packages. The buildverse package itself is a container package that, when installed under devDependencies in your own project, also installs the following buildverse tool suite packages:

Projects using buildverse include:

buildverse-sdk -- yes, the Buildverse SDK uses itself to build itself!
cowarray-develop -- an example of using buildverse in a TypeScript project

Installing

buildverse requires:

Node.js version 4.8.4+
npm version 5.3.0+
npx version 9.6.0+

pnpm version 1.16.0+
yarn version 1.1.0+

buildverse optionally requires:

pnpm version 1.16.0+
yarn version 1.1.0+

Note: npx is a new utility that ships with npm -- see the announcement on The npm Blog — Introducing npx: an npm package runner.

First-time users are advised to read the Tutorial.

Via npx (recommended)

Installation via npx works well on Linux though YMMV (your mileage may vary) on Windows systems.

Using npx, the following command creates a new subdirectory as a multiproject container project in your current working directory. You specify a 'container-project-name' as the name for this directory. buildverse is automatically installed in the devDependencies of your project's package.json file. Command line hints are given on how to create and add subprojects to your multiproject container.

$ npx buildverse create-suite <container-project-name>

Via npm (locally)

The following alternate method requires typing a few more command lines. Some people prefer this method to better understand exactly what is going on.

This method is also a workaround should npx not work well on your system or if you do not otherwise wish to use it.

$ mkdir <container-project-name>
$ cd <container-project-name>
$ npm init -y
$ npm install --save-dev buildverse
$ node_modules/.bin/buildverse init-suite

Tutorial

This is a work in progress but hopefully it is enough for you to gain a basic understanding of installing and using buildverse.

Run the following commands in a terminal session to get a feel for how buildverse installs itself in a new multiproject container project. We'll call this container project 'foobar-develop'.

Initial setup

Command

$ npx buildverse create-suite foobar-develop

Output

Creating multiproject container project 'foobar-develop'.
...
Begin by typing

  `cd foobar-develop`

then create subprojects within this container project with

  `npm run create-project <subproject-name>`

or by other means such as

  `npx create-react-app <subproject-name>`

or by manually creating a subdirectory containing a `package.json` file.

It is also possible to create multiproject container projects nested within
'foobar-develop' with `npm run create-suite <subcontainer-name>` to further
group your subprojects/packages!

Creating & adding subprojects

Here we first change into our container project directory and then create two subprojects 'foo' and 'bar'.

Commands

$ cd foobar-develop
$ npm run create-project foo
$ npm run create-project bar

Output

Creating subproject 'foo'.
...
Add the subproject to the parent container project with
`npm run add-project foo`.

`cd foo` to edit/develop the subproject.

...

Creating subproject 'bar'.
...
Add the subproject to the parent container project with
`npm run add-project bar`.

`cd bar` to edit/develop the subproject.

Next we add the two new subprojects to the container project.

Commands

$ npm run add-project foo
$ npm run add-project bar

Output

Adding subproject 'foo' to parent container.
...
Now create and add more subprojects or start configuring your package.json
scripts!

Adding subproject 'bar' to parent container.
...
Now create and add more subprojects or start configuring your package.json
scripts!

File system picture

After running all of the commands in this tutorial so far, the file system picture -- shown in alphabetical listing order with directories first followed by files -- is this:

foobar-develop/
  bar/
    package.json    # generally list only runtime/production packages in
                    # bar's dependencies and leave out devDependencies
                    # alltogether
  foo/
    package.json    # generally list only runtime/production packages in
                    # foo's dependencies and leave out devDependencies
                    # alltogether
  node_modules/
    ...             # installed devDependencies modules for foobar-develop
  package.json    # list buildverse & other development-time packages in
                  # devDependencies and leave out dependencies alltogether
  package-lock.json

Running container project package scripts

Now, still while in the 'foobar-develop' directory, run the container project's build script:

Command

$ npm run build

What you see in the terminal output is npm running the 'build' script of each of the two subprojects, 'foo' and 'bar'.

Notice that the run order was the same order that the respective projects were added to the 'foobar-develop' multiproject container.

Building all subprojects
(cd foo && npm run build) 

> foo@0.1.0 build .../foobar-develop/foo
> echo "Change this to your subproject's build command"

(cd bar && npm run build) 

> bar@0.1.0 build .../foobar-develop/bar
> echo "Change this to your subproject's build command"

Next run the container project's clean script:

Command

$ npm run clean

This time, surprise! As expected npm runs the 'clean' script for each of the two subprojects though in reverse order.

Cleaning all subprojects
(cd bar && npm run clean) 

> bar@0.1.0 clean .../foobar-develop/bar
> echo "Change this to your subproject's clean command"

(cd foo && npm run clean) 

> foo@0.1.0 clean .../foobar-develop/foo
> echo "Change this to your subproject's clean command"

How does this work?

Examine the package.json files created in the above terminal session.

// foobar-develop/package.json
{
  "name": "foobar-develop",
  "private": true,
  ...
  "scripts": {
    "add-project": "buildverse add-project",
    "build": "echo \"Building all subprojects\" && for-each-project apply-cmdline",
    "clean": "echo \"Cleaning all subprojects\" && for-each-project --reverse apply-cmdline",
    "create-project": "buildverse create-project",
    "create-suite": "buildverse create-suite",
    "posttest": "echo \"All subprojects test scripts exited with code 0\"",
    "pretest": "npm run build",
    "remove-project": "buildverse remove-project",
    "test": "echo \"Testing all subprojects\" && for-each-project apply-cmdline"
  },
  ...
  "buildverse": {
    "subprojects": [
      "foo",
      "bar"
    ]
  }
}

// foobar-develop/foo/package.json
{
  "name": "foo",
  ...
  "scripts": {
    "build": "exit 0",
    "clean": "exit 0",
    "test": "exit 0"
  }
}

// foobar-develop/bar/package.json
{
  "name": "bar",
  ...
  "scripts": {
    "build": "exit 0",
    "clean": "exit 0",
    "test": "exit 0"
  }
}

The core utility in the buildverse tool suite is for-each-project. This utility takes the name of a function (from a set of functions defined elsewhere -- more on that later) as a command line argument. There is a special built-in function called apply-cmdline.

Taken together and when used in an npm package script, for-each-project apply-cmdline does the following things:

Consults the package.json file in the current working directory for a special configuration object under the key buildverse. This object contains an array under the key subprojects which represents a set of subproject directory names.
Each directory entry in this array is then processed by:

Changing the current working directory to the subproject directory
Recursively executing the package manager program (npm, pnpm or yarn) with the same command line arguments that it was originally executed with.

When for-each-project is supplied with the --reverse switch, the subproject directories are processed in reverse order. See the for-each-project package documentation for a number of other flexible command-line options for processing sets of project directories.

Using a text editor you should now be in a position to enhance and expand upon the sample scripts in these package.json files. Simply replace the echo commands with actual commands for invoking tools such as Babel, TypeScript or perhaps just a JS linter in your subproject directories!

This is the bare bones of information you need to start using the buildverse package for developing your project. See the Command-line Usage Docs section for detailed information about all of the commands in the buildverse suite.

Command-line Usage Docs

Support

If you run into any problems or have constructive criticism, suggestions for improvement or just plain encouragement to offer please open or comment on an issue on GitHub.

You can also show your support by following BuildverseJS, @buildverse on Twitter.

License

Currently this project is licensed GPLv3.

The author is working however towards gaining community-funding so as to make this a successful and sustainable, quality open-source project with an MIT license.