# is-web-crawler

> A light weight JS library to check if a user agent is a web crawler.

Latest version **1.1.0** (published 2021-03-24) · MIT license · 0 weekly downloads

## Install

```sh
npm install is-web-crawler
pnpm add is-web-crawler
yarn add is-web-crawler
bun add is-web-crawler
```

## Health

**Score 15/100 (F)** — status: abandoned.

Positive: no vulnerabilities.

Warnings: low downloads; no types; no esm support.

Negative: abandoned; low maintenance score.

## Facts

| | |
|---|---|
| Version | 1.1.0 |
| Published | 2021-03-24 |
| First published | 2021-03-24 |
| Weekly downloads | 0 |
| License | MIT |
| TypeScript types | none |
| Module format | CommonJS |
| Dependencies | 1 |
| Unpacked size | 14.3 KB |
| Known vulnerabilities | 0 |
| Install scripts | no |
| GitHub stars | 1 |
| Author | Prageeth Silva |
| Maintainers | prageethsilva |
| Keywords | web, craweler, js |

## Links

- npm: https://www.npmjs.com/package/is-web-crawler
- Repository: https://github.com/pureartisan/is-web-crawler
- Homepage: https://github.com/pureartisan/is-web-crawler#readme
- Issues: https://github.com/pureartisan/is-web-crawler/issues
- npm.io page: https://npm.io/package/is-web-crawler

## Dependencies (1)

- [crawler-user-agents](https://npm.io/package/crawler-user-agents.md) ^1.0.1

## Alternatives

- [@mapbox/jsonlint-lines-primitives](https://npm.io/package/@mapbox/jsonlint-lines-primitives.md) — 5.3M weekly downloads
- [reftools](https://npm.io/package/reftools.md) — 3.5M weekly downloads
- [@hey-api/openapi-ts](https://npm.io/package/@hey-api/openapi-ts.md) — 3.5M weekly downloads
- [@mapbox/geojson-rewind](https://npm.io/package/@mapbox/geojson-rewind.md) — 2.4M weekly downloads
- [turbo-stream](https://npm.io/package/turbo-stream.md) — 1.7M weekly downloads

## Recent versions

- 1.1.0 (latest) — 2021-03-24
- 1.0.0 — 2021-03-24

## README

# is-web-crawler ![npm](https://img.shields.io/npm/v/is-web-crawler)

A javascript based library to check if the current user agent is a web crawler.
It uses the list of known web crawlers using the package [crawler-user-agents](https://www.npmjs.com/package/crawler-user-agents).

## Get started
```
npm install is-web-crawler
```

## Usage

### Quick Usage

If you want to know if the current user agent is a web crawler.

```
import { isCrawlerUserAgent } from 'is-web-crawler';

const isCurrentUserAgentABot = isCrawlerUserAgent();
```

NOTE: The regex pattern matching under the hood is cached, so calling this method multiple times will not affect performance.

### Only use the top X known crawlers

If you only want to tackle the top X web crawlers (from the list provided by [crawler-user-agents](https://www.npmjs.com/package/crawler-user-agents)), then simply pass in the number as an argument:

```
const topCrawlers = 10;
const isCurrentUserAgentABot = isCrawlerUserAgent(topCrawlers);
```

### Checking a custom user agent string

There might a scenario where you want to check a custom user agent string, instead of the current user agent. In this case, you can use the `isCrawler` method instead.

```
import { isCrawlerUserAgent } from 'is-web-crawler';

const myUserAgent = ... get user agent ...;
const topCrawlers = 10;

// the second argument is optional
const isCurrentUserAgentABot = isCrawler(myUserAgent, topCrawlers);

```
NOTE: This method by passes the caching, so it if you need to cache the value, you need to implement this in your application.

---
_Source: https://npm.io/package/is-web-crawler · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._