@qiwi/repocrawler-cli v1.1.1

@qiwi/repocrawler-cli

CLI for crawling and making reports

Crawler

Usage

yarn add @qiwi/repocrawler-cli
crawler --vcs gerrit --url http://gerrit.com/a --auth.username username --auth.password password --org common --org internal --path package.json

or via npx

npx -p @qiwi/repocrawler-cli crawler --vcs gerrit --url http://gerrit.com/a --auth.username username --auth.password password --org common --path package.json --path .npmrc

You can also use config file:

crawler --config config.json

config.json (you can specify several crawlers):

{
  "org": [
    "common"
  ],
  "crawlers": [
    {
      "vcs": "gerrit",
      "auth": {
        "username": "username",
        "password": "password"
      },
      "url": "https://gerrit.com/a"
    },
    {
      "vcs": "github",
      "auth": "1234567890123456789012345678901234567890",
      "url": "https://github.com/api/v3"
    }
  ]
}

or use config file with overriding:

# override "org" field
crawler --config config.json --org internal

# replace "crawlers" with other one
crawler --config config.json --vcs gerrit --url https://other-gerrit.com/a --auth.username foo --auth.pasword password

Options

vcs, auth, auth.username, auth.password, url should be specified in objects of crawlers array field of config file, see example in Usage. Other flags can be given in config as top-level values. Kebab-case options should be written in camelCase in the config.

Reporter

Usage

yarn add @qiwi/repocrawler-cli
reporter --report tree --cwd temp --package ^typescript$ --deps dev --source package > temp-qiwi-forks-ts-report.json 

or via npx

npx -p @qiwi/repocrawler-cli repoter --report tree --cwd temp --package ^typescript$ --deps dev --source package > temp-qiwi-forks-ts-report.json 

Options

Report types

Tree report

This report type shows for every package satisfying given package pattern:

• list of repositories which use given package;
• the used min and max versions;
• usage count;
• ratio as usageCount/totalProjectsCount.

totalProjectsCount is a number of repositories with package.json.

reporter --report tree --cwd temp --package ^@types --deps dev --source package > temp-qiwi-forks-types-tree-report.json

temp-qiwi-forks-types-tree-report:

{
  "packageNamePattern": "^@types",
  "depType": "dev",
  "source": "package",
  "data": [
    {
      "name": "@types/node",
      "usageCount": 3,
      "ratio": 0.4286,
      "repos": [
        {
          "vcs": "github",
          "repo": "github-qiwi-forks-esm"
        },
        {
          "vcs": "github",
          "repo": "github-qiwi-forks-npm-run-all"
        },
        {
          "vcs": "github",
          "repo": "github-qiwi-forks-npm-types"
        }
      ],
      "minVersion": "14.0.24",
      "maxVersion": "15.0.2"
    }
  ],
  "totalProjectsCount": 7
}

Usage report

This report shows for every package satisfying given package pattern:

• name of npm-project using given package;
• the used version of the package;
• info about the latest commit in the project.

reporter --report usage --cwd temp-qiwi-forks --package ^typescript$ --range ">=4.1.0" --deps dev --source package > temp-qiwi-forks-ts-usage-report.json

temp-qiwi-forks-ts-usage-report.json:

{
  "packageNamePattern": "^typescript$",
  "depType": "dev",
  "source": "package",
  "versionRange": ">=4.1.0",
  "data": [
    {
      "project": "@qiwi/dts-bundle",
      "package": "typescript",
      "versions": [
        "4.2.3"
      ],
      "commitInfo": {
        "hash": "d382c33590a02356ba116ce63f5e79a1ffd85795",
        "message": "chore(release): 0.7.5 [skip ci]\n\n## [0.7.5](https://github.com/qiwi-forks/dts-bundle/compare/v0.7.4...v0.7.5) (2021-03-16)\n\n### Bug Fixes\n\n* **pkg:** up deps, fix vuls ([e230e70](https://github.com/qiwi-forks/dts-bundle/commit/e230e70cc836338a628683f0e4f15f418b967a1a))",
        "date": "2021-03-16T08:17:13Z"
      }
    },
    {
      "project": "@qiwi/npm-types",
      "package": "typescript",
      "versions": [
        "4.3.5"
      ],
      "commitInfo": {
        "hash": "f20f3765f6191133b053743cfdd48db1f92f7997",
        "message": "chore(release): 1.0.3 [skip ci]\n\n## [1.0.3](https://github.com/qiwi-forks/npm-types/compare/v1.0.2...v1.0.3) (2021-07-01)\n\n### Bug Fixes\n\n* add optionalDependencies ([#3](https://github.com/qiwi-forks/npm-types/issues/3)) ([97fa744](https://github.com/qiwi-forks/npm-types/commit/97fa74483715557c320f6987c4eaeb090918af04))\n\n### Performance Improvements\n\n* update deps ([#4](https://github.com/qiwi-forks/npm-types/issues/4)) ([5d0c0f5](https://github.com/qiwi-forks/npm-types/commit/5d0c0f5243bd6c0af6e62685eb0a1cc30bf8bb53))",
        "date": "2021-07-01T16:47:02Z"
      }
    }
  ]
}

Versions report

The same as usage report, but contains used versions only

reporter --report versions --cwd temp-qiwi-forks --package ^typescript$ --range ">=4.1.0" --deps dev --source package > temp-qiwi-forks-ts-versions-report.json

temp-qiwi-forks-ts-versions-report.json:

{
	"packageNamePattern": "^typescript$",
	"depType": "dev",
	"source": "package",
	"versionRange": ">=4.1.0",
	"data": [
		"4.2.3",
		"4.3.5"
	]
}