@runbook/backstage NPM

This is a CLI tool that is being used to experiment with Spotify's backstage

The summarise command

This is intended for CI/CD pipelines. It will scan a directory recursively find the catalog files (anything with a .yaml extension and a apiVersion)

It then summarises the information in the catalog files and outputs it as a json file.

Example Usage

backstage summarise --owner SomeOwner --project SomeProject --repo SomeName --enabled

Produces in the standard output this. Note that --owner, --project, --repo and --enabled are optional. By default enabled is undefined

{
  "owner":    "SomeOwner",
  "project":  "SomeProject",
  "repo": "SomeName",
  "enabled":  true,
  "catalogs": {
    "all":       [.. a list of all catalog files ...],
    "apis":      [.. a list of all catalog files that are defining API ...],
    "services":  [.. a list of all catalog files that are defining services ...],
    "libraries": [.. a list of all catalog files that are defining libraries ...],
    "errors":    [{"file": "somefile.yaml", "error": "some error message because we couldn't parse the file"}]
  }
}

Later this will updated to include information about documents

The make command

It can scan a pom.xml file and generate a backstage catalog for it.

It scans a directory recursively looking for filetypes:

pom.xml - which can have modules in them (or not)
package.json - which can have workspaces in them (or not)
catalog-info.yaml - as long as the first characters are not # Autogenerated
backstage.xxx.yaml - xxx should be the type. i.e. API or service or library. These are when we want to hardcode the information

A location file called catalog-info.yaml is generated at the root. It might point to the poms, or to other location files. It will point to other location files in the case of a monorepo: a location file at the root of each module.

Each pom.xml or package.jsonwill have a catalog-info.xxx.yaml file generated for it. By default it will be added as a library but this can be changed by adding a backstage.kind property to the pom.xml or `package.json

Caveat

This is a work in progress and I am torn whether this should be a backstage plugin or not. My issues with it being a plugin is that the 'developer experience' is not great with plugins: you need to scan the logs a lot, and the error messages are not great. Plus the plugin is 'magic'. Here the files are explicitly generated and the developers are in charge of them, and they can easily be seen.

adding info to pom.xml

We can add properties to the pom.xml file that will be used to generate the catalog-info.yaml file.

Currently the most common properties are:

<properties>
    <backstage.kind>API</backstage.kind>
    <backstage.ignore>true</backstage.ignore>
    <backstage.spec><type>Service</type></backstage.spec>
    <backstage.tags>tag1,tag2</backstage.tags> <!-- comma separated list of tags -->
    <backstage.annotations>key1: value1, key2: value2</backstage.annotations> <!-- comma separated list of key value pairs -->
    <backstage.techdocs>/path/to/where/mkdocs.yml is.</backstage.techdocs> 
</properties>

backstage.kind Which template is used. If not found it uses the default template.
backstage.ignore Whether the module is ignored. This is typically used in modules that are just tests or not published
backstage.spec What type of component are you? The default is library. But you can be a service or an API.
tags Used to give the component tags
annotations Used to give the component annotations
techdocs Used to point to the root of the documentation. Typically this is just .

adding info to package.json

The properties are the same as for pom.xml except that they are provided as follows

{
  "backstage": {
    "kind":        "API",
    "ignore":      true,
    "spec":        {"type": "Service"},
    "tags":        "tag1,tag2",
    "annotations": "key1: value1, key2: value2",
    "techdocs":    "/path/to/where/mkdocs.yml is."
  }
}

Properties and parents and so on

If a pom.xml has a parent that is part of the current 'scan' then the properties are inherited. Same with package.json and workspaces.

This is useful for such things as the repository property. If you have a parent pom.xml that defines the scm, or a parent package.json that defines the repository then you don't need to repeat it in the child pom.xml or package.json

Examples of use

If you have a repo or mono-repo with one or more pom.xml or package.json files then you can use this tool to generate the catalog-info.yaml files. It will add in the dependencies between 'local' modules (i.e. the ones under the directory that you are scanning).

backstage --help
backstage catalog make --owner nameOfTheOwner
backstage catalog make --owner nameOfTheOwner -d /path/to/where/the/pom.xml/files/are

If you get an error like the following, it means that you have a module that doesn't have a name. Typically this would be a package.json file that is the root of yarn workspaces. To fix this you can add a name to the package.json file or use the --name someName option to the command line.

Errors
No name for xxx
No name for root

The rollup command

This gathers statistics about the catalog files. It is intended to be used in a CI/CD pipeline.

backstage rollup myorganisation

This uses the premise that in my azure devops organisation there is a special project called DevHub (configuratable). In this project there is a repo also called DevHub with a file in it which holds a list of projects and who owns them.

Then in each of those azure devops projects there is a special repo. Default name is DevHub but this is configurable. This repo has a file in it which is a list of repos in the project with a boolean flag to say whether it is enabled or not.

Then in each of those repos is a stats.txt file. This holds the information generated by the backstage summarise command.

This command gathers all the stats.txt files and puts them into a json structure which is sent to the standard output.

The purpose of this is to allow stats to be places in the individual repos and then gathered together in a central place.

The fileApi command

This is a simple command that turns serves the files in the current directory (and below) as an API. It is intended to be used in a CI/CD pipeline when validating the catalog-info.yaml files. It is needed because the backstage api can only accept a url for a location and during the CI/CD pipeline we don't have a url for the location without this.

```bash

# installation

```bash
npm install -g @runbook/backstage

Example usages

backstage --help
backstage catalog make --name Rest2 --owner phil-rice -l production

Debugging

The debug menu is mostly to help the developers of the cli tool. However it has some useful things for anyone

Nuke

If you want to get rid of all the autogenerated files then you can use the nuke command. This removes all .yaml files that have # Autogeneated at the start

backstage debug nuke

files

Shows all the catalog-info.npm.yaml and catalog-info.maven.yaml files that would be generated.

backstage debug files

locations

This is the mirror of debug files. It just shows the location files that would be generated. i.e. the catalog-info.yaml that points to the files
shown by debug files

backstage debug locations
backstage debug locations --content

With --content you can see the content of the files

arrays

This is about the parent child relationships between pom.xml or package.json.

backstage debug arrays

docs

Seaches for subdirectories of the main project directories called 'docs' and the files 'mkdocs.yml'. This is just for information. To add the docs you need to modify the pom.xml or package.json file.

backstage debug docs

vars

If you are making and debugging your own templates, then you might want to see what are the 'variables' that the template has access to. I find it helpful to pipe this to a file and then look at that file in a text editor