# fs-async-crawler

> Asynchronous options for crawling file system

Latest version **0.0.3** (published 2019-07-25) · ISC license · 0 weekly downloads

## Install

```sh
npm install fs-async-crawler
pnpm add fs-async-crawler
yarn add fs-async-crawler
bun add fs-async-crawler
```

## Health

**Score 15/100 (F)** — status: abandoned.

Positive: no vulnerabilities.

Warnings: low downloads; no types; no esm support; pre 1.0.

Negative: abandoned; low maintenance score.

## Facts

| | |
|---|---|
| Version | 0.0.3 |
| Published | 2019-07-25 |
| First published | 2019-07-25 |
| Weekly downloads | 0 |
| License | ISC |
| TypeScript types | none |
| Module format | CommonJS |
| Dependencies | 2 |
| Unpacked size | 21.7 KB |
| Known vulnerabilities | 0 |
| Install scripts | no |
| Author | Ryan Vazquez |
| Maintainers | ryanvazquez |
| Keywords | fs, read, directory, crawl |

## Links

- npm: https://www.npmjs.com/package/fs-async-crawler
- Repository: https://github.com/ryanvazquez/fs-crawler
- Homepage: https://github.com/ryanvazquez/read-dir#readme
- Issues: https://github.com/ryanvazquez/read-dir/issues
- npm.io page: https://npm.io/package/fs-async-crawler

## Dependencies (2)

- [chalk](https://npm.io/package/chalk.md) ^2.4.2
- [micromatch](https://npm.io/package/micromatch.md) ^4.0.2

## Alternatives

- [unionfs](https://npm.io/package/unionfs.md) — 2.2M weekly downloads
- [path-starts-with](https://npm.io/package/path-starts-with.md) — 35.9K weekly downloads
- [redzip](https://npm.io/package/redzip.md) — 1.2K weekly downloads
- [vscode-anymatch](https://npm.io/package/vscode-anymatch.md) — 848 weekly downloads
- [@ledgerhq/coin-filecoin](https://npm.io/package/@ledgerhq/coin-filecoin.md) — 793 weekly downloads

## Recent versions

- 0.0.3 (latest) — 2019-07-25
- 0.0.2 — 2019-07-25
- 0.0.1 — 2019-07-25

## README

[![Build Status](https://travis-ci.org/ryanvazquez/fs-async-crawler.svg?branch=master)](https://travis-ci.org/ryanvazquez/fs-async-crawler)

# fs-async-crawler
  File system crawler and async API

## **Install**
```sh
  npm install fs-async-crawler
```
  
  ### **Config**
  | Property | Type | Description | Default | Required |
  | :------- | :--- | :---------- | :------ | :------ |
  | `root` | `string` | **Absolute** path of the directory to crawl | `null`| true
  | `maxDepth` | `number` |  A number representing the maximum depth of recursion. The crawler will terminate the crawling of a branch once the maxDepth has been reached. **Warning:** If not specified, the crawler will crawl the **entire** directory tree. This could have signficant performance impact. | `null` | false
  | `ignorePaths`| `array <rgx>` | An array of regular expressions. The crawler will ignore any directories and any files that match to provided paths. | `[/node_modules/]`| false
  | `match` | `array <glob>` | An array of glob strings. If provided, the crawler will collect only the files whose paths match the provided globs. | `null` | false
  | `strict` | `boolean` | Determines whether the crawler throw an error when trying to access a file the parent process does not have permissions for.  | `false` | `false`

    
# **API**
## Crawler.prototype.all

### Crawls entire directory starting from the root. Returns an array of absolute filePaths.

> crawler.all(finalCallback)

quick setup: 
```js
const Crawler = require('fs-async-crawler');
const crawler = new Crawler({ root: 'Users/path/to/files' });

// gets all files within root
crawler.all((err, files) => {
  if (err){
    console.log('There was an error! ' + err);
  } else {
    console.log(files);
  }
});
```

with options:

```js
const Crawler = require('fs-async-crawler');
const config = {
  root: 'Users/path/to/dir',
  ignorePaths: [/node_modules/, /__tests__/],
  match: ['**.js']
}
const crawler = new Crawler(config);

// get all JS files within root
crawler.all((err, files) => {
  if (err){
    console.log('There was an error! ' + err);
  } else {
    console.log(files);
  }
});
```
       
## Crawler.prototype.forEach

#### Crawls the directory and performs an async operation on each file.

Like Array.prototype.forEach, Crawler.prototype.forEach executes the async function but does not return/collect the result. The `done` callback accepts an error as its only argument.

> crawler.forEach(iteratee, finalCallback);

```js
const Crawler = require('fs-async-crawler');
const config = {
  root: 'Users/path/to/log-files',
  ignorePaths: [/node_modules/],
  match: ['**.log']
}
const crawler = new Crawler(config);

// get all log files within root and delete them
crawler.forEach((file, done) => {

  deleteFile(file, (err, result) => {
    if (err){
      return done(err);
    }
    return done(null);
  })

}, (err) => {
  if (err){
    console.log('Something went wrong');
  } else {
    console.log('Everything ok!');
  }
});
```

## Crawler.prototype.map

### Crawls the directory, performs an async operation on each file and collects the result in an array. 

Results are not guaranteed to be in their original order.

> crawler.map(iteratee, finalCallback);

```js
const Crawler = require('fs-async-crawler');
const config = {
  root: 'Users/path/to/dir',
  match: ['**.txt']
}
const crawler = new Crawler(config);

// find all txt files within root and create an array of their contents
crawler.map((file, done) => {

  getContentsOfTxtFile(file, (err, content) => {
    if (err){
      return done(err, null);
    }
    return done(null, content);
  })

}, (err, contents) => {
  if (err){
    console.log('There was an error!' + err);
  } else {
    console.log(contents);
  }
});
```


## Crawler.prototype.reduce

### Crawls the directory, performs an async operation on each file and returns an accumulated result.

Note: Reduce requires the accumulated result of each previous operation to be passed to the current operation. Therefore, async tasks must be executed serially, rather than in parallel (the default behavior of Crawler). This may have an adverse impact on performance.

> crawler.reduce(initialValue, iteratee, finalCallback);


```js
const Crawler = require('fs-async-crawler');
const config = {
  root: 'Users/path/to/dir',
  match: ['**.txt']
};
const crawler = new Crawler(config);
const initialValue = 0;

crawler.reduce(initialValue, (acc, file, done) => {

  readFileContents(file, (err, content) => {
    if (err){
      return done(err, null);
    }
    return done(null, acc + content.length);
  })

}, (err, contentLength) => {
  if (err){
    console.log('An error occured!' + err);
  } else {
    console.log('Content length of all files' + contentLength);
  }
});
```

---
_Source: https://npm.io/package/fs-async-crawler · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._