# xenforo-dl

> XenForo Forum Downloader

Latest version **1.0.0** (published 2024-03-15) · MIT license · 0 weekly downloads

## Install

```sh
npm install xenforo-dl
pnpm add xenforo-dl
yarn add xenforo-dl
bun add xenforo-dl
```

Provides the command `xenforo-dl`.

## Health

**Score 25/100 (F)** — status: abandoned.

Positive: has types; esm support; no vulnerabilities.

Warnings: low downloads.

Negative: abandoned; low maintenance score.

## Facts

| | |
|---|---|
| Version | 1.0.0 |
| Published | 2024-03-15 |
| First published | 2024-03-15 |
| Weekly downloads | 0 |
| License | MIT |
| TypeScript types | bundled |
| Module format | ESM |
| Node | >=16.16.0 |
| Dependencies | 16 |
| Unpacked size | 250.7 KB |
| Known vulnerabilities | 0 |
| Install scripts | no |
| GitHub stars | 16 |
| Author | Patrick Kan |
| Maintainers | patrickkfkan |
| Keywords | xenforo, download, downloader, scrape, scraper |

## Links

- npm: https://www.npmjs.com/package/xenforo-dl
- Repository: https://github.com/patrickkfkan/xenforo-dl
- Homepage: https://github.com/patrickkfkan/xenforo-dl#readme
- Issues: https://github.com/patrickkfkan/xenforo-dl/issues
- npm.io page: https://npm.io/package/xenforo-dl

## Dependencies (16)

- [cheerio](https://npm.io/package/cheerio.md) ^1.0.0-rc.12
- [fs-extra](https://npm.io/package/fs-extra.md) ^9.1.0
- [cli-color](https://npm.io/package/cli-color.md) ^2.0.3
- [fast-copy](https://npm.io/package/fast-copy.md) ^3.0.1
- [bottleneck](https://npm.io/package/bottleneck.md) ^2.19.5
- [dateformat](https://npm.io/package/dateformat.md) ^5.0.3
- [node-fetch](https://npm.io/package/node-fetch.md) ^3.3.2
- [deep-freeze](https://npm.io/package/deep-freeze.md) ^0.0.1
- [prompt-sync](https://npm.io/package/prompt-sync.md) ^4.2.0
- [html-to-text](https://npm.io/package/html-to-text.md) ^9.0.5
- [trim-newlines](https://npm.io/package/trim-newlines.md) ^5.0.0
- [@types/cli-color](https://npm.io/package/@types/cli-color.md) ^2.0.5
- [command-line-args](https://npm.io/package/command-line-args.md) ^5.2.1
- [sanitize-filename](https://npm.io/package/sanitize-filename.md) ^1.6.3
- [command-line-usage](https://npm.io/package/command-line-usage.md) ^7.0.1
- [content-disposition](https://npm.io/package/content-disposition.md) ^0.5.4

## Alternatives

- [@tsparticles/shape-image](https://npm.io/package/@tsparticles/shape-image.md) — 303.7K weekly downloads
- [@tsparticles/shape-line](https://npm.io/package/@tsparticles/shape-line.md) — 233.7K weekly downloads
- [stringify-attributes](https://npm.io/package/stringify-attributes.md) — 58.6K weekly downloads
- [mobile-drag-drop](https://npm.io/package/mobile-drag-drop.md) — 46.3K weekly downloads
- [@comunica/actor-rdf-parse-html](https://npm.io/package/@comunica/actor-rdf-parse-html.md) — 29.2K weekly downloads

## Recent versions

- 1.0.0 (latest) — 2024-03-15

## README

<a href='https://ko-fi.com/C0C5RGOOP' target='_blank'><img height='36' style='border:0px;height:36px;' src='https://storage.ko-fi.com/cdn/kofi2.png?v=3' border='0' alt='Buy Me a Coffee at ko-fi.com' /></a>

# xenforo-dl

A [XenForo](https://xenforo.com/) forum downloader written in [Node.js](https://nodejs.org):

- Scrapes content from forum pages
- For each thread, downloads attachments and saves messages in text files
- Supports downloading a single thread or all threads in a forum
- Supports continuing from previous download

Since the downloader works through scraping, it is not guaranteed to work with all XenForo forums. I created the downloader for my data-hoarding needs targeting a handful of sites, so it might be limited in what it can scrape. But feel free to raise issues.


## Installation

First, install [Node.js](https://nodejs.org/).

Then, in a terminal, run the following command:

```
npm i -g xenforo-dl
```

## Usage

```
$ xenforo-dl [OPTION]... URL
```

### URL

#### Thread URLs

Pattern: `<forum_site_url>/threads/<title_slug>.<thread_id>[/page-<num>]`

Download all messages and attachments shown on page. If content spans multiple pages, download from subsequent pages as well.

If `page-<num>` is present in URL, then download will begin with the specified page.

#### Forum URLs

Pattern: `<forum_site_url>/forums/<title_slug>.<forum_id>[/page-<num>]`

Download all threads listed on page. If the forum has threads spanning multiple pages, download from subsequent pages as well.

If `page-<num>` is present in URL, then download will begin with the specified page.

#### Other URLs

For URLs not matching the above patterns, `xenforo-dl` will scrape for forum links and download from them. It is your responsibility to ensure the given URL is a valid XenForo link.

### Options

| Option    | Description |
|-----------|-------------|
| `-h`, `--help` | Display usage guide |
| `-k`, `--cookie` | (string) Cookie to set in requests. See [Cookies](#cookies). |
| `-o`, `--out-dir` | (string) Path of save directory. Default: current working directory. |
| `-d`, `--dir-structure` | Combination of flags controlling the output directory structure of downloaded threads: <ul><li>`s`: Include directory for the forum site.</li><li>`pl`: Include directory for each category or forum leading up to the target thread.</li><li>`pi`: Include directory for the immediate section or forum containing the target thread.</li><li>`t`: Include directory for the target thread itself.</li><li>`a`: Include directory for attachments.</li><li>`-`: No directory structure. Everything will be saved directly to --out-dir.</li></ul><p>Default: `splta`</p>|
| `-w`, `--overwrite` | Overwrite existing attachment files |
| `-l`, `--log-level` | Log level: `info`, `debug`, `warn` or `error`; set to `none`` to disable logging. Default: `info` |
| `-s`, `--log-file` | (string) Save logs to specified path |
| `-r`, `--max-retries` | (number) Maximum retry attempts when a download fails. Default: 3 |
| `-c`, `--max-concurrent`| (number) Maximum number of concurrent downloads for attachments. Default: 10 |
| `-p`, `--min-time-page` | (number) Minimum time, in milliseconds, to wait between page fetch requests. Default: 500 |
| `-i`, `--min-time-image` | (number) Minimum time, in milliseconds, to wait between download requests for attachments. Default: 200 |
| `--continue` | Continue from previous download |
| `-y`, `--no-prompt` | Do not prompt for confirmation to proceed |

### Cookies

Cookies allow you to download content that would otherwise be inaccessible due to lack of user credentials. To obtain a cookie for passing to `xenforo-dl` through the `--cookie` option, do the following:

1. In a browser, sign in to the target forum site.
2. Press `F12` to bring up Developer Tools.
3. Select `Network` tab, followed by `HTML` filter.
4. Press `F5` to refresh the page. Select one of the entries that appear under the `Network` tab.
5. Under `Headers` -> `Request Headers`, you should see the `Cookie` entry. Copy the value of that entry and pass it to `xenforo-dl`.

Cookies should remain valid until they expire or you sign out of the forum site.

## Changelog

v1.0.0
- Initial release

## License

MIT

---
_Source: https://npm.io/package/xenforo-dl · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._