# robots-txt-guard

> Validate urls against robots.txt rules.

Latest version **1.0.2** (published 2021-02-08) · MIT license · 0 weekly downloads

## Install

```sh
npm install robots-txt-guard
pnpm add robots-txt-guard
yarn add robots-txt-guard
bun add robots-txt-guard
```

## Health

**Score 15/100 (F)** — status: abandoned.

Positive: no vulnerabilities.

Warnings: low downloads; no types; no esm support.

Negative: abandoned; low maintenance score.

## Facts

| | |
|---|---|
| Version | 1.0.2 |
| Published | 2021-02-08 |
| First published | 2014-10-14 |
| Weekly downloads | 0 |
| License | MIT |
| TypeScript types | none |
| Module format | CommonJS |
| Dependencies | 0 |
| Unpacked size | 413.1 KB |
| Known vulnerabilities | 0 |
| Install scripts | no |
| Author | Jan Potoms |
| Maintainers | woorank-admin, janpotoms, ndemoor, woorank-ci |

## Links

- npm: https://www.npmjs.com/package/robots-txt-guard
- Repository: https://github.com/Woorank/robots-txt-guard
- Homepage: https://github.com/Woorank/robots-txt-guard#readme
- Issues: https://github.com/Woorank/robots-txt-guard/issues
- npm.io page: https://npm.io/package/robots-txt-guard

## Recent versions

- 1.0.2 (latest) — 2021-02-08
- 1.0.1 — 2021-02-07
- 1.0.0 — 2021-02-07
- 0.2.1 — 2017-12-04
- 0.1.1 — 2017-11-29
- 0.1.0 — 2014-12-07
- 0.0.2 — 2014-10-29
- 0.0.1 — 2014-10-14

## README

# robots-txt-guard [![Build Status](https://travis-ci.org/Woorank/robots-txt-guard.svg)](https://travis-ci.org/Woorank/robots-txt-guard)

Validate urls against robots.txt rules. Configure with output from [robots-txt-parse](https://github.com/Woorank/robots-txt-parse/)

## Usage

```js
var guard = require('robots-txt-guard');

var robotsTxt = guard({
  groups: [{
    agents: [ '*' ],
    rules: [
      { rule: 'allow', path: '/' }
    ]
  }, {
    agents: [ 'googlebot', 'twitterbot' ],
    rules: [
      { rule: 'disallow', path: '/tmp/*' },
      { rule: 'disallow', path: '/temporary/*' },
      { rule: 'noindex', path: '/temporary/*' }
    ]
  }]
});

robotsTxt.isAllowed('Googlebot', '/tmp/abc'); // false
robotsTxt.isAllowed('mozilla', '/tmp/abc'); // true
robotsTxt.isAllowed('googlebot-news', '/home.html'); // true

robotsTxt.isIndexable('googlebot', '/tmp/*'); // true
robotsTxt.isIndexable('googlebot', '/temporary/*'); // false
```

---
_Source: https://npm.io/package/robots-txt-guard · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._