1.1.6 • Published 6 years ago

github-resource-converter v1.1.6

Weekly downloads
2
License
MIT
Repository
github
Last release
6 years ago

github-resource-converter NPM version

Convert GitHub Issues and Pull Requests to JSON and CSV from a Terminal or Node.js app.

The MIT License FOSSA Status NSP Status Dependency Status Development Dependency Status MacOS and Ubuntu build statuses Windows build status Coverage percentage Codacy code quality NPMS score NPM downloads per month

Table of contents

1. Overview

github-resource-converter (alias grc) exports your GitHub and GitHub Enterprise repositories' Issues and Pull Requests to CSV and JSON file formats. It's helpful whenever you need to:

  • Use spreadsheets to analyze, modify, print, or summarize large amounts of data with Pivot Tables and other important financial or statistical operations

    Example:

    # Convert all GitHub issues to CSV:
$ github-resource-converter \
  --owner foo \
  --repo  bar
# => Saved "foo-bar-issues-export.2018-03-20T02_11_04_356Z.csv".

  • Share data with other tools like GitLab and JIRA.

    Example:

    # Convert and save all GitHub Enterprise
# Pull Requests as JSON (using the grc alias):
$ grc \
  --owner gregswindle \
  --repo github-resource-converter \
  --resource-type pr \
  --dest './docs/reports/export.json'
# Saved "docs/reports/gregswindle-github-resource-converter-pr-export.2018-03-20T02_18_33_682Z.json".

2. Installation

  1. Required: github-resource-converter is written in JavaScript (CommonJS) for Node.js External link version

which must be installed prior to use. Node.js requires npm, which is used for installing dependencies. (npm installs with Node.js.)

  1. Recommended: To avoid rate-limiting, you should create a personal access token External link and save your personal access token.

    • MacOS and Unix

      $ mkdir -p /usr/local/etc/github-resource-center/envvars/
$ touch /usr/local/etc/github-resource-center/envvars/.env
$ echo "GITHUB_ACCESS_TOKEN="{your-personal-access-token-value}" > \
  /usr/local/etc/github-resource-center/envvars/.env

    • Windows

      > md -p C:\usr\local\etc\github-resource-center\envvars\
> touch C:\usr\local\etc\github-resource-center\envvars\.env
> echo "GITHUB_ACCESS_TOKEN="{your-personal-access-token-value}" >>
  C:\usr\local\etc\github-resource-center\envvars\.env

2.1. For Terminal/command-line usage

# Install globally to execute from a Terminal/command-line
$ npm i -g github-resource-converter

2.2. As a application dependency

# Install as a dependency within a Node.js app
$ npm i --save github-resource-converter

3. Usage

The following examples assume that github-resource-converter is installed globally and invoked from a Terminal (command-line interface)

3.1. Formatting

--dest export.[csv|json]

You can convert GitHub (Enterprise) Issues and Pull Requests into two file formats: CSV and JSON.

  • CSV is the default format.
  • JSON formatting requires a --dest value with a .json file extension.

3.2. Resource types

--resource-type, -t

Convert and export GitHub (Enterprise) by --resource-type: issues, pull_requests, or both (all).

  • issues is the default resource-type.

  • prs require a --resource-type or -t value of

    • pr
    • prs
    • pull_request
    • pull_requests

  • --resource-type all will export all issues and prs into a single file.

3.3. Filtering

alert Filtering is currently unavailable.

If you're interested in CONTRIBUTING to features like filters--e.g., only select "open" issues--we're happily accepting pull requests!

3.4. Exporting

3.4.1. Issues

  • CSV

    GitHub:

    # GET https://api.github.com/rrepos/:owner/:repo/issues
$ github-resource-converter \
  --owner gregswindle \
  --repo  github-resource-converter \
  --dest  './docs/reports/export.csv'

    GitHub Enterprise:

    # GET https://api.ecorp.com/api/v3/repos/:owner/:repo/issues
$ grc --base-url https://api.ecorp.com/api/v3 \
  --owner evilcorp \
  --repo ecoin

  • JSON

    GitHub:

    # GET https://api.github.com/repos/:owner/:repo/issues
$ github-resource-converter \
  --owner gregswindle \
  --repo  github-resource-converter \
  --dest  './docs/reports/export.json'

    GitHub Enterprise:

    # GET https://api.ecorp.com/api/v3/repos/:owner/:repo/issues
$ grc --base-url https://api.ecorp.com/api/v3 \
  --owner evilcorp \
  --repo ecoin
  --dest ./export.json

3.4.2. Pull Requests

  • CSV

    GitHub:

    # GET https://api.github.com/repos/:owner/:repo/pulls
$ github-resource-converter \
  --owner gregswindle \
  --repo  github-resource-converter \
  --resource-type pr

    GitHub Enterprise:

    # GET https://api.ecorp.com/api/v3/repos/:owner/:repo/pulls
$ grc --base-url https://api.ecorp.com/api/v3 \
  --owner evilcorp \
  --repo ecoin \
  --resource-type pr

  • JSON

    GitHub:

    # GET https://api.github.com/repos/:owner/:repo/pulls
$ github-resource-converter \
  --owner gregswindle \
  --repo  github-resource-converter \
  --resource-type pr
  --dest './export.json'

    GitHub Enterprise:

    # GET https://api.ecorp.com/api/v3/repos/:owner/:repo/pulls
$ grc --base-url https://api.ecorp.com/api/v3 \
  --owner evilcorp \
  --repo ecoin \
  --resource-type pr
  --dest './export.json'

3.4.3. All (open and closed issues and pull requests)

  • CSV

    GitHub:

    # GET https://api.github.com/repos/:owner/:repo/pulls
$ github-resource-converter \
  --owner gregswindle \
  --repo  github-resource-converter \
  --resource-type all

  • GitHub Enterprise:

    # GET https://api.ecorp.com/api/v3/repos/:owner/:repo/pulls
$ grc --base-url https://api.ecorp.com/api/v3 \
  --owner evilcorp \
  --repo ecoin \
  --resource-type all

  • JSON

    GitHub:

    # GET https://api.github.com/repos/:owner/:repo/pulls
$ github-resource-converter \
  --owner gregswindle \
  --repo  github-resource-converter \
  --resource-type all
  --dest './export.json'

  • GitHub Enterprise:

    # GET https://api.ecorp.com/api/v3/repos/:owner/:repo/pulls
$ grc --base-url https://api.ecorp.com/api/v3 \
  --owner evilcorp \
  --repo ecoin \
  --resource-type all
  --dest './export.json'

3.5. Command-line flags

3.6. Errors

Errors are written to the console (stdout) as JSON:

# Attempt to fetch issues from a repository that doesn't exist
$ grc --owner example --repo error
[2018-03-20T02:31:24.737Z] ERROR: github-resource-converter/48219 on User.router.home: {"message":"Not Found","documentation_url":"https://developer.github.com/v3/issues/#list-issues-for-a-repository"}
  HttpError: {"message":"Not Found","documentation_url":"https://developer.github.com/v3/issues/#list-issues-for-a-repository"}
      at response.text.then.message (/p/a/t/h/github-resource-converter/node_modules/@octokit/rest/lib/request/request.js:56:19)
      at <anonymous>
      at process._tickCallback (internal/process/next_tick.js:188:7)

3.7. Info

The --help flag displays all options:

$ grc --help

  Convert and export GitHub resources--Issues and Pull Requests--to CSV and JSON.

  Usage

    $ grc [options] [info]
    $ github-resource-converter [options] [info]

  Options

    --base-url           The GitHub REST API v3 URL origin, or a
                         GitHub Enterprise URL origin and path-prefix.
                         [Default: 'https://api.github.com']
    --dest,          -d  The CSV's destination path and file name.
                         [Default: './resources.csv']
    --no-auto-filename   Disable automatic file naming.
                         [Default: false]
    --owner,         -o  The GitHub account name or organization name.
    --repo,          -r  The name of the GitHub (or GitHub enterprise)
                         repository.
    --resource-type, -t  "issues", "prs", or "all".
                         [Default: 'issues']

  Info

    --help     Show this dialog.
    --version  Display the installed semantic version.

  Examples

    $ grc --owner github --repo hub
      // => Exported CSV to /path/of/cwd/issues.csv.

    $ grc --owner github --repo hub -dest './reports/issues/YYYY-MM-DD.csv'
      // => Exported CSV to /path/to/reports/issues/YYYY-MM-DD.csv.

    $ grc --owner example --repo error
      // =>
      [2018-03-19T08:04:06.596Z] ERROR: github-resource-converter/00000 on localhost: Cannot destructure property `data` of 'undefined' or 'null'.
        TypeError: Cannot destructure property `data` of 'undefined' or 'null'.
            at paginate (/p/a/t/h/github-resource-converter/lib/base-resource-converter.js:39:16)
            at <anonymous>
            at process._tickCallback (internal/process/next_tick.js:188:7

Use the --version flag to see which version you have installed:

$ github-resource-converter --version
# => 1.0.1

4. API

beaker Test the github-resource-converter (grc) API in your Web browser link-external.

4.1. grc.authenticate({token, type, key}): void

Info Most GitHub API calls don't require authentication. Rules of thumb:

  1. If you can see the information by visiting the site without being logged in, you don't have to be authenticated to retrieve the same information through the API.
  2. If you want to change data, you have to be authenticated.

octokit/rest.js. (2018). GitHub. Retrieved 21 March 2018, from https://github.com/octokit/rest.js#authentication link-external

4.1.1. Parameters

NameTypeDescriptionNotes
keyString
tokenString
typeEnumbasic, oauth, oauth-key-secret, token, and integration

4.1.2. Example

// Token (https://github.com/settings/tokens)
grc.authenticate({
  token: 'secrettoken123',
  type: 'token'
})

4.2. grc.getAll({owner, repo}): Promise

Retrieve all open and closed Issues and Pull Requests from a GitHub project.

GET

/repos/:owner/:repo/issues
/repos/:owner/:repo/pulls

info grc.getAll combines the results of grc.issues.getForRepo and grc.pullRequests.getForRepo into a single Array<object>.

For more information, see

  1. grc.issues.getForRepo and
  2. grc.pullRequests.getForRepo below.

4.3. grc.issues.getForRepo({owner, repo}): Promise

Retrieve all open and closed issues for a repository with this proxy method for octokit.issues.getForRepo link-external.

GET

/repos/:owner/:repo/issues

4.3.1. Parameters

4.3.2. Examples

  • async/await:

    const result = await grc.issues.getForRepo({
  owner: 'gregswindle',
  repo: 'eslint-plugin-crc'
})

  • Promise:

    grc.issues
  .getForRepo({
    owner: 'gregswindle',
    repo: 'eslint-plugin-crc'
  })
  .then(result => {})
  .catch(err => {})

4.4. grc.logger

A trentm/node-bunyan link-external logger instance that uses the LONG thlorenz/bunyan-format link-external writeable stream for output.

info bunyan.INFO is the default log LEVEL.

Logger {
  domain: null,
  _events: {},
  _eventsCount: 0,
  _maxListeners: undefined,
  _level: 30,
  streams:
    [{
      type: 'stream',
      stream: BunyanFormatWritable {
      },
      closeOnExit: false,
      level: 30,
      raw: false
    }],
  serializers: null,
  src: false,
  fields: {
    name: 'github-resource-converter',
     hostname: 'Gregorys-MBP-2.fios-router.home',
    pid: 76698
  },

  debug(): void,
  error(): void,
  fatal(): void,
  info(): void,
  trace(): void,
  warn(): void
}

4.5. grc.options

Contains default values for api, cli, and metadata.

{
  "api": {
    "baseUrl": "https://api.github.com",
    "headers": {
      "Accept":
        "application/vnd.github.v3+json, application/vnd.github.symmetra-preview+json",
      "user-agent": "gregswindle/github-resource-converter v1.0.1"
    },
    "owner": null,
    "repo": null,
    "requestMedia": "application/vnd.github.v3+json",
    "timeout": 0
  },
  "cli": {
    "flags": {
      "baseUrl": {
        "default": "https://api.github.com",
        "type": "string"
      },
      "dest": {
        "alias": "d",
        "default": "./export.csv",
        "type": "string"
      },
      "no-auto-filename": {
        "default": false,
        "type": "boolean"
      },
      "owner": {
        "alias": "o",
        "type": "string"
      },
      "repo": {
        "alias": "r",
        "type": "string"
      },
      "resource-type": {
        "alias": "t",
        "default": "issues",
        "type": "string"
      }
    }
  },
  "meta": {
    "input": [],
    "flags": {
      "noAutoFilename": false,
      "baseUrl": "https://api.github.com",
      "dest": "./export.csv",
      "d": "./export.csv",
      "resourceType": "issues",
      "t": "issues"
    },
    "pkg": {
      "name": "github-resource-converter",
      "description":
        "Convert and export GitHub resources--Issues and Pull Requests--to CSV and JSON.",
      "version": "1.0.1",
      "author": {
        "name": "Greg Swindle",
        "email": "greg@swindle.net",
        "url": "https://github.com/gregswindle"
      },
      "bin": {
        "grc": "lib/cli.js"
      },
      "bugs": {
        "url": "https://github.com/gregswindle/github-resource-converter/issues"
      },
      "commitplease": {
        "style": "angular",
        "types": [
          "build",
          "chore",
          "ci",
          "docs",
          "feat",
          "fix",
          "perf",
          "refactor",
          "revert",
          "style",
          "test"
        ],
        "scope": "\\S+.*"
      },
      "contributors": [],
      "dependencies": {
        "@octokit/rest": "15.2.6",
        "bunyan": "1.8.12",
        "bunyan-format": "0.2.1",
        "dotenv": "5.0.1",
        "fs-extra": "5.0.0",
        "insight": "0.10.1",
        "jsonexport": "2.0.11",
        "lodash": "4.17.5",
        "meow": "4.0.0"
      },
      "devDependencies": {
        "@semantic-release/changelog": "2.0.1",
        "@semantic-release/git": "4.0.1",
        "@semantic-release/npm": "3.2.4",
        "ajv": "6.4.0",
        "ajv-keywords": "3.1.0",
        "codacy-coverage": "2.1.1",
        "commitplease": "3.2.0",
        "coveralls": "3.0.0",
        "eslint": "4.19.1",
        "eslint-config-prettier": "^2.4.0",
        "eslint-config-scanjs": "1.0.0-beta4",
        "eslint-config-standard": "11.0.0",
        "eslint-config-xo": "0.20.1",
        "eslint-plugin-import": "2.10.0",
        "eslint-plugin-jsdoc": "3.5.0",
        "eslint-plugin-json": "1.2.0",
        "eslint-plugin-no-unsafe-innerhtml": "1.0.16",
        "eslint-plugin-node": "6.0.1",
        "eslint-plugin-prettier": "^2.2.0",
        "eslint-plugin-promise": "3.7.0",
        "eslint-plugin-security": "1.4.0",
        "eslint-plugin-standard": "3.0.1",
        "eslint-plugin-unicorn": "4.0.2",
        "eslint-plugin-xss": "0.1.9",
        "fixpack": "2.3.1",
        "husky": "^0.14.3",
        "jest": "22.4.3",
        "jest-runner-eslint": "0.4.0",
        "lec": "^1.0.1",
        "lint-staged": "7.0.3",
        "markdown-magic": "0.1.21",
        "markdown-magic-dependency-table": "1.3.2",
        "markdown-magic-install-command": "1.3.1",
        "markdown-magic-package-scripts": "1.2.1",
        "nsp": "^3.2.1",
        "prettier": "1.11.1",
        "semantic-release": "15.1.5"
      },
      "engines": {
        "node": ">=8.0.0"
      },
      "eslintIgnore": ["lib/__tests__/coverage/**"],
      "files": ["lib"],
      "homepage":
        "https://github.com/gregswindle/github-resource-converter/#readme",
      "jest": {
        "automock": false,
        "collectCoverage": true,
        "coverageDirectory": "lib/__tests__/coverage",
        "coverageThreshold": {
          "global": {
            "branches": 100,
            "functions": 100,
            "lines": 100,
            "statements": 100
          }
        },
        "moduleDirectories": ["node_modules", "lib"],
        "testMatch": [
          "<rootDir>/lib/__tests__/**/*.test.js",
          "<rootDir>/lib/__tests__/*.test.js"
        ],
        "watchman": false
      },
      "jest-runner-eslint": {
        "cliOptions": {
          "config": ".github/config/jest-runner-eslint.config.json",
          "fix": true
        }
      },
      "keywords": [
        "QA",
        "convert",
        "converter",
        "converter",
        "csv",
        "export",
        "github",
        "google sheets",
        "issue",
        "json",
        "pr",
        "pull request",
        "pull-request",
        "quality assurance",
        "quality-assurance",
        "reporting",
        "reports",
        "save",
        "sheets",
        "spreadsheets",
        "testing"
      ],
      "license": "MIT",
      "lint-staged": {
        "*.js": ["npm run lint:js", "git add"],
        "*.json": ["npm run lint:json", "git add"],
        "*.md": ["npm run lint:md", "npm run docs", "git add"]
      },
      "main": "lib/index.js",
      "prettier": {
        "semi": false,
        "singleQuote": true
      },
      "release": {
        "generateNotes": {
          "preset": "angular"
        },
        "prepare": [
          "@semantic-release/changelog",
          "@semantic-release/npm",
          "@semantic-release/git"
        ],
        "verifyConditions": ["@semantic-release/npm", "@semantic-release/git"]
      },
      "repository": {
        "type": "git",
        "url":
          "git+https://github.com/gregswindle/github-resource-converter.git"
      },
      "scripts": {
        "docs": "npm run docs:toc",
        "docs:toc":
          "md-magic --config '.github/config/markdown.config.js' --path '**/*.md' --ignore 'node_modules'",
        "lint":
          "npm run lint:js && npm run lint:json && npm run lint:manifest && npm run lint:md",
        "lint:js":
          "node_modules/.bin/eslint -c .github/config/.eslintrc.yml --ext .js . --fix",
        "lint:json":
          "prettier ./**/*.json --ignore-path '.github/config/.prettierignore' --write",
        "lint:manifest": "fixpack",
        "lint:md": "prettier ./**/*.md -prose-wrap always --write",
        "posttest:ci:coverage:codacy":
          "cat ./lib/__tests__/coverage/lcov.info | codacy-coverage",
        "precommit": "lint-staged",
        "prepare": "npm run lint && npm run security",
        "prepublish":
          "lec lib/cli.js -c LF && npm run security && semantic-release -d",
        "prepublishOnly": "npm run prepare",
        "pretest": "npm run lint",
        "security": "npm run security:nsp:scan",
        "security:nsp:scan": "nsp check",
        "test": "jest",
        "test:config": "jest --showConfig",
        "test:watch": "jest ./lib/__tests__/*.test.js --watch",
        "test:watch:all": "jest ./lib/__tests__/*.test.js --watchAll"
      },
      "readme": "ERROR: No README data found!",
      "_id": "github-resource-converter@1.0.1"
    },
    "help":
      "\n  Convert and export GitHub resources--Issues and Pull Requests--to CSV and JSON.\n\n  Usage\n\n    $ grc [options] [info]\n    $ github-resource-converter [options] [info]\n\n  Options\n    --base-url           The GitHub REST API v3 URL origin, or a\n                         GitHub Enterprise URL origin and path-prefix.\n                         [Default: 'https://api.github.com']\n    --dest,          -d  The CSV's destination path and file name.\n                         [Default: './resources.csv']\n    --no-auto-filename       Don't append an ISO 8601-like timestamp to the\n         output file.\n                         [Default: false]\n    --owner,         -o  The GitHub account name or organization name.\n    --repo,          -r  The name of the GitHub (or GitHub enterprise)\n                         repository.\n    --resource-type, -t  \"issues\", \"prs\", or \"all\".\n                         [Default: 'issues']\n\n  Info\n\n    --help     Show this dialog.\n    --version  Display the installed semantic version.\n\n  Examples\n\n    $ grc --owner github --repo hub\n      // => Exported CSV to /path/of/cwd/issues.csv.\n\n    $ grc --owner github --repo hub -dest './reports/issues/YYYY-MM-DD.csv'\n      // => Exported CSV to /path/to/reports/issues/YYYY-MM-DD.csv.\n\n    $ grc --owner example --repo error\n      // =>\n      [2018-03-19T08:04:06.596Z] ERROR: github-resource-converter/00000 on localhost: Cannot destructure property `data` of 'undefined' or 'null'.\n        TypeError: Cannot destructure property `data` of 'undefined' or 'null'.\n            at paginate (/p/a/t/h/github-resource-converter/lib/base-resource-converter.js:39:16)\n            at <anonymous>\n            at process._tickCallback (internal/process/next_tick.js:188:7)\n"
  }
}

4.6. grc.pullRequests.getForRepo({owner, repo}): Promise

Retrieve an array of all open and closed pull requests for a GitHub or GitHub Enterprise repository with this proxy method for octokit.pullRequests.getAll link-external.

GET

/repos/:owner/:repo/pulls

4.6.1. Parameters

4.6.2. Examples

  • async/await:

    const grc = require('github-resource-coverter')

const getAllPullRequests = async (params) = {
  try {
    const WHITESPACE = 2
    const prs = await grc.pullRequests.getForRepo(params)
    grc.logger.info(JSON.stringify(prs, null, WHITESPACE))
  } catch (err) {
    grc.logger.error(err)
  }
}

await getAllPullRequests({
  owner: 'democracy-ia',
  repo: 'govinfo-link-js'
})

  • Promises:

    const grc = require('github-resource-coverter')

grc.pullRequests
  .getForRepo({
    owner: 'democracy-ia',
    repo: 'govinfo-link-js'
  })
  .then(prs => {
    const WHITESPACE = 2
    grc.logger.info(JSON.stringify(prs, null, WHITESPACE))
  })
  .catch(err => {
    grc.logger.error(err)
  })

4.7. grc.save({data, dest}): Promise

Export a collection of Issues or Pull Requests to your local filesystem.

4.7.1. Parameters

4.7.2. Examples

  • async/await:

    // Save as JSON
const result = await grc.save({
  data,
  dest: './export.json'
})

  • Promise:

    // Save as CSV
grc
  .save({
    data,
    dest: './export.csv'
  })
  .then(result => {})
  .catch((err = {}))

4.8. grc.toCsv({data=[]}): Promise

Converts (deeply) nested JSON into CSV format, returning a Promise<string>.

4.8.1. Parameters

4.8.2. Examples

  • async/await:

    const result = await grc.toCsv(data)

  • Promise:

    grc
  .toCsv(data)
  .then(result => {})
  .catch((err = {}))

5. Version

NPM version

6. Contributing

PRs Welcome External link We welcome contributions with GitHub issues and pull requests.

Request a feature Report a defect

Read the CONTRIBUTING guidelines

Before embarking on a significant change, please adhere to the following guidelines:

  1. Create an issue—e.g., a defect ("bug") report or a feature request—to propose changes.

    Exceptions:

    If you're working on documentation and fixing something simple like a typo or an easy bug, go ahead and make a pull request.

  2. Follow the CONTRIBUTING guidelines.

    Why:

    Standards and guidelines make communication easier. If you're willing and able to program—or want to learn how— following the guidelines will increase the likelihood of adding your changes to the software product.

  3. Read the Code of Conduct.

    Why:

    It's more fun when everybody's friendly and respectful.

  4. Make a pull request when you're ready for other to review your changes (or you get stuck somewhere).

    PR novices:

    🙋 Never created a pull request? No problem. 🆓 Take this free online training External link. (It even covers most of the conventions in the CONTRIBUTING guidelines!)

7. License

MIT © Greg Swindle.

Read the NOTICE External link for all third-party software that github-resource-converter uses.

FOSSA Status

Greenkeeper badge Readme Score

QAconvertconverterconvertercsvexportgithubgoogle sheetsissuejsonprpull requestpull-requestquality assurancequality-assurancereportingreportssavesheetsspreadsheetstesting
@octokit/restbunyanbunyan-formatdotenvfs-extrainsightjsonexportlodash.isstringmeow
@everything-registry/sub-chunk-1766@zalastax/nolb-github-r
1.1.6

6 years ago

1.1.5

6 years ago

1.1.4

6 years ago

1.1.3

6 years ago

1.1.2

6 years ago

1.1.1

6 years ago

1.1.0

6 years ago

1.0.1

6 years ago

1.0.0

6 years ago

1.0.0-alpha

6 years ago