# sitemap-crawler

> Generate sitemap just throw any link.

Latest version **1.0.0** (published 2018-08-11) · MIT license · 0 weekly downloads

## Install

```sh
npm install sitemap-crawler
pnpm add sitemap-crawler
yarn add sitemap-crawler
bun add sitemap-crawler
```

## Health

**Score 15/100 (F)** — status: abandoned.

Positive: no vulnerabilities.

Warnings: low downloads; no types; no esm support.

Negative: abandoned; low maintenance score.

## Facts

| | |
|---|---|
| Version | 1.0.0 |
| Published | 2018-08-11 |
| First published | 2018-08-11 |
| Weekly downloads | 0 |
| License | MIT |
| TypeScript types | none |
| Module format | CommonJS |
| Dependencies | 4 |
| Unpacked size | 6.8 KB |
| Known vulnerabilities | 0 (+1 in 1 direct dependencies) |
| Install scripts | no |
| Author | tinyjin |
| Maintainers | tinyjin |
| Keywords | sitemap, crawler, href |

## Links

- npm: https://www.npmjs.com/package/sitemap-crawler
- Repository: https://github.com/tinyjin/sitemap-crawler
- Issues: https://github.com/tinyjin/sitemap-crawler/issues
- npm.io page: https://npm.io/package/sitemap-crawler

## Dependencies (4)

- [async](https://npm.io/package/async.md) ^2.6.1
- [cheerio](https://npm.io/package/cheerio.md) ^1.0.0-rc.2
- [request](https://npm.io/package/request.md) ^2.88.0
- [progress](https://npm.io/package/progress.md) ^2.0.0

## Recent versions

- 1.0.0 (latest) — 2018-08-11

## README

# Sitemap Crawler
Generate sitemap just throw any link.

## Intro
``sitemap-crawler`` collect directly accessible url through resolve with href value.
## Basic Usage
```js
const siteMap = require('sitemap-crawler');
const link = 'http://www.npmjs.com';

siteMap(link, (err, res) => {
  console.log('error:', err);
  console.log('siteMap:', res); // Print the siteMap from link
});
```
**Result**
```json
[
  "https://npmjs.com/features",
  "https://npmjs.com/pricing",
  "https://npmjs.com/support",
  "https://npmjs.com/signup",
  "https://npmjs.com/signup?next=/org/create",
  "https://npmjs.com/get-npm",
  "https://npmjs.com/enterprise",
  ...
]
```

## Plural Link
You can crawl from string array that includes link.

In this case, crawler response **object** type.
```js
const siteMap = require('sitemap-crawler');
const links = [
  'http://www.npmjs.com',
  'http://github.com',
  'www.amazon.com'
]

siteMap(links, (err, res) => {
  console.log('error:', err);
  console.log('siteMap:', res); // Print the siteMap from link
});
```
**Result**
```json
{
  "count": 3,
  "siteMap": {
    "http://www.npmjs.com": [...],
    "http://www.amazon.com": [...],
    "http://github.com": [...]
  }
}
```

## Options
You can use prepared options.
- ``isProgress`` ``Boolean`` : If true, show CLI Progress while crawl.
- ``isLog`` ``Boolean`` : If true, print request error log.
```js
const siteMap = require('sitemap-crawler');
const link = 'http://www.npmjs.com';

siteMap(link, {isProgress : true, isLog : true}, (err, res) => {
  console.log('error:', err);
  console.log('siteMap:', res); // Print the siteMap from link
});
```

## Authors
tinyjin - [Github](https:github.com/tinyjin), [Blog](https://wlsdml1103.blog.me)

## License
This project has MIT License.

---
_Source: https://npm.io/package/sitemap-crawler · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._