# spidex

> A web crawler for node.js, even support hessian request.

Latest version **3.0.1** (published 2024-10-18) · MIT license · 0 weekly downloads

## Install

```sh
npm install spidex
pnpm add spidex
yarn add spidex
bun add spidex
```

## Health

**Score 35/100 (D)** — status: abandoned.

Positive: has types; no vulnerabilities; high quality score.

Warnings: low downloads; no esm support.

Negative: abandoned.

## Facts

| | |
|---|---|
| Version | 3.0.1 |
| Published | 2024-10-18 |
| First published | 2014-01-17 |
| Weekly downloads | 0 |
| License | MIT |
| TypeScript types | bundled |
| Module format | CommonJS |
| Dependencies | 7 |
| Unpacked size | 48.6 KB |
| Known vulnerabilities | 0 |
| Install scripts | no |
| GitHub stars | 25 |
| Author | XadillaX |
| Maintainers | xadillax |
| Keywords | clawler, hessian |

## Links

- npm: https://www.npmjs.com/package/spidex
- Repository: https://github.com/XadillaX/spidex
- Homepage: https://github.com/XadillaX/spidex#readme
- Issues: https://github.com/XadillaX/spidex/issues
- npm.io page: https://npm.io/package/spidex

## Dependencies (7)

- [lodash](https://npm.io/package/lodash.md) ^4.17.21
- [urlencode](https://npm.io/package/urlencode.md) ^2.0.0
- [hessian.js](https://npm.io/package/hessian.js.md) ^2.11.0
- [iconv-lite](https://npm.io/package/iconv-lite.md) ^0.6.3
- [stream-http](https://npm.io/package/stream-http.md) ^3.2.0
- [eventemitter3](https://npm.io/package/eventemitter3.md) ^5.0.1
- [https-browserify](https://npm.io/package/https-browserify.md) ^1.0.0

## Recent versions

- 3.0.1 (latest) — 2024-10-18
- 3.0.0 — 2024-10-17
- 2.2.1 — 2017-03-31
- 2.2.0 — 2017-03-31
- 2.1.2 — 2016-09-08
- 2.1.1 — 2015-11-06
- 2.1.0 — 2015-11-04
- 2.0.5 — 2015-08-08
- 2.0.4 — 2015-02-28
- 2.0.3 — 2015-02-21
- 2.0.2 — 2015-02-17
- 2.0.1 — 2014-12-11
- 2.0.0 — 2014-12-11
- 1.1.1 — 2014-12-09
- 1.1.0-rdm — 2014-12-09
- … 8 more at https://npm.io/package/spidex/versions

## README

# Spidex

[![npm version](https://img.shields.io/npm/v/spidex.svg)](https://npmjs.org/spidex)
[![npm downloads](https://img.shields.io/npm/dm/spidex.svg)](https://npmjs.org/spidex)
[![Build Status](https://github.com/XadillaX/spidex/workflows/Node.js%20CI/badge.svg)](https://github.com/XadillaX/spidex/actions)
[![Coverage Status](https://img.shields.io/coveralls/XadillaX/spidex/master.svg)](https://coveralls.io/github/XadillaX/spidex?branch=master)

Spidex is a versatile web requester for Node.js and browsers, designed to simplify HTTP requests with a clean and
intuitive API.

## Features

- Supports both Node.js and browser environments.
- Handles GET, POST, PUT, DELETE and other methods.
- Customizable request options including headers, timeouts, and charsets.
- Built-in support for various character encodings.
- Hessian v2 protocol support (Node.js only).
- Event-based error handling.

## Installation

Install Spidex using npm:

```bash
npm install spidex --save
```

## Usage

### Basic Request

```js
const spidex = require('spidex');

spidex.get('https://api.example.com/data', (content, statusCode, responseHeaders) => {
  console.log('Response:', content);
  console.log('Status:', statusCode);
  console.log('Headers:', responseHeaders);
}).on('error', err => {
  console.error('Error:', err);
});
```

### Request with Options

```js
spidex.post('https://api.example.com/users', {
  data: JSON.stringify({ username: 'john_doe', email: 'john@example.com' }),
  header: { 'Content-Type': 'application/json' },
  charset: 'utf8',
  timeout: 5000
}, (content, statusCode, responseHeaders) => {
  console.log('User created:', content);
}).on('error', (err) => {
  console.error('Error creating user:', err);
});
```

### Supported Basic HTTP Methods

Spidex supports the following HTTP methods:

- `spidex.get(url, [options], [callback])`
- `spidex.post(url, [options], [callback])`
- `spidex.put(url, [options], [callback])`
- `spidex.delete(url, [options], [callback])`

Each method returns an EventEmitter that emits an 'error' event if an error occurs.

### Supported Other HTTP Methods

- `spidex.method(url, method, [options], [callback])`

### Request Options

The `options` object can include the following properties:

- `data`: Request body (string, object, or Buffer)
- `header`: Custom request headers
- `charset`: Character encoding (e.g., 'utf8', 'gbk', 'binary', etc.)
- `timeout`: Total request timeout in milliseconds
- `responseTimeout`: Response timeout in milliseconds
- `requestTimeout`: Request timeout in milliseconds

### Parsing Cookies

Spidex provides a utility function to parse cookies from response headers:

```js
const cookies = spidex.parseCookies(responseHeaders);
console.log('Parsed cookies:', cookies);
```

### User Agent Management

You can get or set the default User-Agent string:

```js
// Get the current User-Agent
const currentUA = spidex.getDefaultUserAgent();

// Set a custom User-Agent
spidex.setDefaultUserAgent('MyApp/1.0');
```

### Hessian v2 Support (Node.js only)

Spidex supports Hessian v2 protocol for Node.js environments:

```js
spidex.hessianV2('http://hessian.example.com/api', 'methodName', [arg1, arg2], (err, result) => {
  if (err) {
    console.error('Hessian request failed:', err);
    return;
  }
  console.log('Hessian result:', result);
});
```

## TypeScript Support

Spidex includes TypeScript definitions. You can import and use it in TypeScript projects:

```ts
import * as spidex from 'spidex';

spidex.get('https://api.example.com/data', (content, statusCode, responseHeaders) => {
  console.log('Typed response:', content);
});
```

## Error Handling

All Spidex methods return an EventEmitter that emits an 'error' event. You can handle errors by listening to this event:

```js
spidex.get('https://api.example.com/data')
  .on('error', (err) => {
    console.error('Request failed:', err);
  });
```

## License

Spidex is released under the MIT License. See the [LICENSE](LICENSE) file for details.

---
_Source: https://npm.io/package/spidex · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._