Repo2file NPM | npm.io

CircleCI

Repo 2 File

This module helps save a repository as a file. There are three available formats: '.txt', '.md', and '.pdf'. Additionally, there's a method to convert all downloaded repositories into a single file.

This module focuses on raw downloading with a minimalist data structure. Various configuration options can be customized by overwriting the config file, see #config.

Designed for AI Embedding Generation

Quickstart

terminal

npm init -y && npm i repo2file

node index.mjs

import { Repo2File } from 'repo2file'

const repo2file = new Repo2File()

await repo2file.single( {
    'userName': 'EasyMina',
    'repository': 'easyMina',
    'branch': 'main',
    'pdf': false,
    'txt': false,
    'md': true
} )

terminal

node index.mjs

Repo 2 File

Methods

single()

This method allows you to download a repository and save it in the desired format.

Parameters

No.	Key	Description	Required	Type	Default
1	`userName`	GitHub username	X	`string`
2	`repository`	GitHub repository	X	`string`
3	`branch`	Repository branch		`string`	"main"
4	`pdf`	Enable PDF output		`boolean`	false
5	`txt`	Enable plain text output		`boolean`	false
6	`md`	Enable Markdown output		`boolean`	true

To generate output in .pdf or .txt format, the respective key must be included.

Example

import { Repo2File } from 'repo2file'

const repo2file = new Repo2File()

await repo2file.single( {
    'userName': 'EasyMina',
    'repository': 'easyMina',
    'branch': 'main',
    'pdf': true,
    'txt': true,
    'md': true
} )

batch()

This method expects an array of single() objects. It can be useful when you want to download repositories on a specific topic and later use merge() to combine them.

Parameters

Each item requires the following key/value pairs.

No.	Key	Description	Required	Type	Default
1	`cmds`	An array of commands	X	`array of objects`
4	`pdf`	Enable PDF output		`boolean`	false
5	`txt`	Enable plain text output		`boolean`	false
6	`md`	Enable Markdown output		`boolean`	true

The cmds key/value pair expects an array of objects. These objects must include the following key/value pairs:

No.	Key	Description	Required	Type	Default
1	`userName`	GitHub username	X	`string`
2	`repository`	GitHub repository	X	`string`
3	`branch`	Repository branch		`string`	"main"

Example

import { Repo2File } from 'repo2file'

const repo2file = new Repo2File()

await repo2file.batch( {
    'cmds': [
        {
            'userName': 'a6b8',
            'repository': 'mina-ns',
            'branch': 'main'
        },
        {
            'userName': 'EasyMina',
            'repository': 'easyMina',
            'branch': 'main'
        }
    ],
    'pdf': true,
    'txt': true,
    'md': true
} )

merge()

This method copies all downloaded repositories into individual files. The default formats are 'pdf', 'markdown', and plain text.

*Parameters

No.	Key	Description	Required	Type	Default
1	`name`	This name will be inserted at the beginning of the eventual filename	X	`string`

import { Repo2File } from 'repo2file'

const repo2file = new Repo2File()

await repo2file.merge( {
    'name': 'my-collection'
} )

Config

All module settings are stored in a config file, see ./src/data/config.mjs. This file can be completely overridden by passing an object during initialization.

import { Repo2File } from 'repo2file'

const myOwnConfig = {
    'meta': {
        'unix': '<<UNIX>>', 
        'format': '<<DATE FORMAT>>'
    },
    'path': {
        'root': './repo2file-temp',
        'route': './repo2file-temp/{{userName}}/{{repository}}/{{branch}}/',
        'raw': 'raw/',
        'pdf': ''
    },
    'merge': {
        'root': './repo2file-temp',
        'outputFormat': '{{name}}--{{timestamp}}.{{suffix}}'
    },
    'output': {
        'pdf': {
            'use': true,
            'outputFormat': '{{userName}}--{{repository}}--{{branch}}.pdf'
        },
        'markdown': {
            'use': true,
            'outputFormat': '{{userName}}--{{repository}}--{{branch}}.md'
        },
        'txt': {
            'use': true,
            'outputFormat': '{{userName}}--{{repository}}--{{branch}}.txt'
        }
    },
    'github': {
        'download': 'https://github.com/{{userName}}/{{repository}}/archive/refs/heads/{{branch}}.zip',
        'root': 'https://github.com'
    },
    'markdown': {
        'code': '~~~',
        'file': {
            'open': '<file>',
            'close': '</file>'
        },
        'repo': {
            'open': '<repository>',
            'close': '</repository>'
        },
        'replaces': [
            [ "`", "&nbsp;" ]
        ],
        'splitSection': false,
        'markdownEscape': false,
        'fileName': 'https://github.com/{{userName}}/{{repository}}/blob/{{branch}}'
    },
    'excludes': {
        'fileNames': [
            'licence',
            'license',
            'code_of_conduct.md',
            '.gitignore',
            '.gitmodules',
            'package-lock.json',
            'yarn.lock',
            '.git',
            '.vscode',
            '.idea',
            '.vs',
            'node_modules',
            '.ds_store'
        ],
        'suffix': [
            //'.md',
            '.png',
            '.yml',
            '.jpg',
            '.jpeg',
            '.gif',
            '.svg',
            '.bmp',
            '.webp',
            '.ico',
            '.mp4',
            '.mov',
            '.avi',
            '.wmv',
            '.pdf',
            '.zip'
        ]
    },
    'console': {
        'space': 50
    }
}

const repo2file = new Repo2File( myOwnConfig )

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/a6b8/repo2file. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the code of conduct.