patreon-dl v1.3.0

patreon-dl

A Patreon downloader written in Node.js.

Features

• Access to patron-only content through cookie
• Download posts by user, in a collection or single post
• Download products (aka shop purchases)
• Items included in downloads:
  • videos
  • images
  • audio
  • attachments
• Save campaign and content info
• Extensively configurable

You can run patreon-dl from the command-line or use it as a library for your project. Node.js v16.16.0 or higher required.

Limitations

• Embedded videos, other than those from YouTube, are not supported (e.g. Vimeo). Only info about these embeds is saved.
• Likewise, embedded links are not followed; only info about the embed is saved.

FFmpeg dependency

FFmpeg is required when downloading:

• videos that are provided only in streaming format; and
• embedded YouTube videos.

Not all video downloads require FFmpeg, but you should have it installed on your system anyway.

Embedded YouTube videos - Premium access

patreon-dl supports downloading embedded YouTube videos. In addition, if you have a YouTube Premium subscription, you can connect patreon-dl to your account and download videos at qualities available only to Premium accounts (e.g. '1080p Premium'). For CLI users, you would configure patreon-dl as follows:

$ patreon-dl --configure-youtube

For library usage, see Configuring YouTube connection.

Installation

npm i -g patreon-dl

The -g option is for installing patreon-dl globally and have the CLI executable added to the PATH. Depending on your usage, you might not need this.

CLI usage

$ patreon-dl [OPTION]... URL

OPTION

URL

Supported URL formats

// Download a product
https://www.patreon.com/<creator>/shop/<slug>-<product_id>

// Download posts by creator
https://www.patreon.com/<creator>/posts

// Dowload a single post
https://www.patreon.com/posts/<slug>-<post_id>

// Download posts in a collection
https://www.patreon.com/collection/<collection_id>

Multiple URLs

You may specify multiple URLs by separating them with a comma. E.g.:

// First download posts by johndoe, followed by posts by janedoe.
$ patreon-dl "https://www.patreon.com/johndoe/posts,https://www.patreon.com/janedoe/posts"

Supplying URLs through file

You can also use a file to supply URLs to patreon-dl. For example, you can have a urls.txt that has the following content:

# Each URL is placed in its own line
# Comments (lines starting with '#') will be ignored

https://www.patreon.com/johndoe/posts
https://www.patreon.com/janedoe/posts

You can then pass urls.txt to patreon-dl:

$ patreon-dl urls.txt

Directory structure

Content is saved with the following directory structure:

out-dir
    ├── campaign
        ├── campaign_info
        ├── posts
        │   ├── post 1
        │   │   ├── post_info
        │   │   ├── images
        │   │   ├── ...
        │   ├── post 2
        │       ├── post_info
        │       ├── images
        │       ├── ...
        ├──shop
            ├── product 1
                ├── product_info
                ├── content_media
                ├── ...

Configuration file

Command-line options are limited. To access the full range of options, create a configuration file and pass it to patreon-dl with the (capital) -C option.

Refer to the example config to see what options are offered. Also see How to obtain Cookie.

Note that you can override an option from a configuration file with one provided at the command-line, provided of course that a command-line equivalent is available.

Library usage

To use patreon-dl in your own project:

import PatreonDownloader from 'patreon-dl';

const url = '....';

const downloader = await PatreonDownloader.getInstance(url, [options]);

await downloader.start();

Here, we first obtain a downloader instance by calling PatreonDownloader.getInstance(), passing to it the URL we want to download from (one of the supported URL formats) and downloader options, if any.

Then, we call start() on the downloader instance to begin the download process. The start() method returns a Promise that resolves when the download process has ended.

• To monitor status and progress, see Workflow and Events.
• To abort a download process, see Aborting.

Downloader options

An object with the following properties (all optional):

Campaign directory name format

Format to apply when naming campaign directories. A format is a string pattern consisting of fields enclosed in curly braces.

What is a campaign directory?

A format must contain at least one of the following fields:

• creator.vanity
• creator.name
• creator.id
• campaign.name
• campaign.id

Characters enclosed in square brackets followed by a question mark denote conditional separators. If the value of a field could not be obtained or is empty, the conditional separator immediately adjacent to it will be omitted from the name.

Default: '{creator.vanity} - ?{campaign.name}' Fallback: 'campaign-{campaign.id}'

Content directory name format

Format to apply when naming content directories. A format is a string pattern consisting of fields enclosed in curly braces.

What is a content directory?

A format must contain at least one of the following unique identifier fields:

• content.id: ID of content
• content.slug: last segment of the content URL

In addition, a format can contain the following fields:

• content.name: post title or product name
• content.type: type of content ('product' or 'post')
• content.publishDate: publish date (ISO UTC format)

Characters enclosed in square brackets followed by a question mark denote conditional separators. If the value of a field could not be obtained or is empty, the conditional separator immediately adjacent to it will be omitted from the name.

Default: '{content.id} - ?{content.name}' Fallback: '{content.type}-{content.id}'

Media filename format

Filename format of a downloaded item. A format is a string pattern consisting of fields enclosed in curly braces.

A format must contain at least one of the following fields:

• media.id: ID of the item downloaded (assigned by Patreon)
• media.filename: can be one of the following, in order of availability:
  • original filename included in the item's API data; or
  • filename derived from the header of the response to the HTTP download request.

In addition, a format can contain the following fields:

• media.type: type of item (e.g. 'image' or 'video')
• media.variant: where applicable, the variant of the item (e.g. 'original', 'thumbnailSmall'...for images)

If media.variant is not included in the format, it will be appended to it if allMediaVariants is true.

Sometimes media.filename could not be obtained, in which case it will be replaced with media.id, unless it is already present in the format.

Characters enclosed in square brackets followed by a question mark denote conditional separators. If the value of a field could not be obtained or is empty, the conditional separator immediately adjacent to it will be omitted from the name.

Default: '{media.filename}' Fallback: '{media.type}-{media.id}'

Configuring YouTube connection

In its simplest form, the process of connecting patreon-dl to a YouTube account is as follows:

1. Obtain credentials by having the user visit a Google page that links his or her account to a 'device' (which in this case is actually patreon-dl).
2. Save the credentials, as a JSON string, to a file.
3. Pass the path of the file to PatreonDownloader.getInstance()

To obtain credentials, you can use the YouTubeCredentialsCapturer class:

import { YouTubeCredentialsCapturer } from 'patreon-dl';

// Note: you should wrap the following logic inside an async
// process, and resolve when the credentials have been saved.

const capturer = new YouTubeCredentialsCapturer();

/**
 * 'pending' event emitted when verification data is ready and waiting
 * for user to carry out the verification process.
 */
capturer.on('pending', (data) => {
  // `data` is an object: { verificationURL: <string>, code: <string> }
  // Use `data` to provide instructions to the user:
  console.log(
    `In a browser, go to the following Verification URL and enter Code:

    - Verification URL: ${data.verificationURL}
    - Code: ${data.code}

    Then wait for this script to complete.`);
});

/**
 * 'capture' event emitted when the user has completed verification and the 
 * credentials have been relayed back to the capturer.
 */
capturer.on('capture', (credentials) => {
  // `credentials` is an object which you need to save to file as JSON string.
  fs.writeFileSync('/path/to/yt-credentials.json', JSON.stringify(credentials));
  console.log('Credentials saved!');
});

// When you have added the listeners, start the capture process.
capturer.begin();

Then, pass the path of the file to PatreonDownloader.getInstance():

const downloader = await PatreonDownloader.getInstance(url, {
  ...
  pathToYouTubeCredentials: '/path/to/yt-credentials.json'
});

You should ensure the credentials file is writable, as it needs to be updated with new credentials when the current ones expire. The process of renewing credentials is done automatically by the downloader.

Logger

Logging is optional, but provides useful information about the download process. You can implement your own logger by extending the Logger abstract class:

import { Logger } from 'patreon-dl';

class MyLogger extends Logger {

  log(entry) {
    // Do something with log entry
  }

  // Called when downloader ends, so you can
  // clean up the logger process if necessary.
  end() {
    // This is not an abstract function, so you don't have to
    // implement it if there is no action to be taken here. Default is
    // to resolve right away.
    return Promise.resolve();
  }
}

Each entry passed to log() is an object with the following properties:

• level: info, debug, warn or error, indicating the severity of the log message.
• originator: (string or undefined) where the message is coming from.
• message: array of elements comprising the message. An element can be anything such as a string, Error or object.

Built-in loggers

The patreon-dl library comes with the following Logger implementations that you may utilize:

• ConsoleLogger

  Outputs messages to the console:

  import { ConsoleLogger } from 'patreon-dl';
  
  const myLogger = new ConsoleLogger([options]);
  
  const downloader = await PatreonDownloader.getInstance(url, {
      ...
      logger: myLogger
  });

  options: (object)

  Option	Description
  enabled	Whether to enable this logger. Default: true
  logLevel	info, debug, warn or error. Default: infoOutput messages up to the specified severity level.
  include	What to include in log messages: (object)dateTime: show date / time of log messages. Default: truelevel: show the severity level. Default: trueoriginator: show where the messsage came from. Default: trueerrorStack: for Errors, whether to show the full error stack. Default: false
  dateTimeFormat	The pattern to format data-time strings, when include.dateTime is true.Date-time formatting is provided by dateformat library. Refer to the README of that project for pattern rules.Default: 'mmm dd HH:MM:ss'

• FileLogger

  Like ConsoleLogger, but writes messages to file.

  import { FileLogger } from 'patreon-dl';
  
  const myLogger = new FileLogger(init, [options]);
  
  const downloader = await PatreonDownloader.getInstance(url, {
      ...
      logger: myLogger
  });

  init: values that determine the name of the log file (object)

  Property	Description
  targetURL	The url passed to PatreonDownloader.getInstance()
  outDir	Value of outDir specified in PatreonDownloader.getInstance() options, or undefined if none specified (in which case defaults to current working directory).
  date	(optional) Date instance representing the creation date / time of the logger. Default: current date-timeYou might want to provide this if you are creating multiple FileLogger instances and filenames are to be formatted with the date, otherwise the date-time part of the filenames might have different values.

  options: all ConsoleLogger options plus the following:

  Option	Description
  logDir	Path to directory of the log file.The path can be a string pattern consisting of the following fields enclosed in curly braces:out.dir: value of outDir provided in init (or the default current working directory if none provided).target.url.path: the pathname of targetURL provided in init, sanitized as necessary.datetime.<date-time format>: the date-time of logger creation, as represented by date in init and formatted according to <date-time format> (using pattern rules defined by the dateformat library).
  logFilename	Name of the log file.The path can be a string pattern consisting of the following fields enclosed in curly braces:target.url.path: the pathname of targetURL provided in init, sanitized as necessary.datetime.<date-time format>: the date-time of logger creation, as represented by date in init and formatted according to <date-time format> (using pattern rules defined by the dateformat library).Default: '{datetime.yyyymmdd}-{log.level}.log'
  fileExistsAction	What to do if log file already exists? One of the following values: append: append logs to existing fileoverwrite: overwrite the existing fileDefault: append

• ChainLogger

  Combines multiple loggers into one single logger.

  import { ConsoleLogger, FileLogger, ChainLogger } from 'patreon-dl';
  
  const consoleLogger = new ConsoleLogger(...);
  const fileLogger = new FileLogger(...);
  const chainLogger = new ChainLogger([ consoleLogger, fileLogger ]);
  
  const downloader = await PatreonDownloader.getInstance(url, {
      ...
      logger: chainLogger
  });

Aborting

To prematurely end a download process, use AbortController to send an abort signal to the downloader instance.

const downloader = await PatreonDownloader.getInstance(...);
const abortController = new AbortController();
downloader.start({
    signal: abortController.signal
});

...

abortController.abort();

// Downloader aborts current and pending tasks, then ends.

Workflow and Events

Workflow

1. Downloader analyzes given URL and determines what targets to fetch.
2. Downloader begins fetching data from Patreon servers. Emits fetchBegin event.
3. Downloader obtains the target(s) from the fetched data for downloading.
4. For each target (which can be a campaign, product or post):
   1. Downloader emits targetBegin event.
   2. Downloader determines whether the target needs to be downloaded, based on downloader configuration and target info such as accessibility.
      • If target is to be skipped, downloader emits targetEnd event with isSkipped: true. It then proceeds to the next target, if any.
   3. If target is to be downloaded, downloader saves target info (subject to downloader configuration), and emits phaseBegin event with phase: saveInfo. When done, downloader emits phaseEnd event.
   4. Downloader begins saving media belonging to target (again, subject to downloader configuration). Emits phaseBegin event with phase: saveMedia.
      1. Downloader saves files that do not need to be downloaded, e.g. embedded video / link info.
      2. Downloader proceeds to download files (images, videos, audio, attachments, etc.) belonging to the target in batches. For each batch, downloader emits phaseBegin event with phase: batchDownload. When done, downloader emits phaseEnd event with phase: batchDownload.
         • In this phaseBegin event, you can attach listeners to the download batch to monitor events for each download. See Download Task Batch.
   5. Downloader emits phaseEnd event with phase: saveMedia.
   6. Downloader emits targetEnd event with isSkipped: false, and proceeds to the next target.
5. When there are no more targets to be processed, or a fatal error occurred, downloader ends with end event.

Events

const downloader = await PatreonDownloader.getInstance(...);

downloader.on('fetchBegin', (payload) => {
    ...
});

downloader.start();

Each event emitted by a PatreonDownloader instance has a payload, which is an object with properties containing information about the event.

Download Task Batch

Files are downloaded in batches. Each batch is provided in the payload of phaseBegin event with phase: batchDownload. You can monitor events of individual downloads in the batch as follows:

downloader.on('phaseBegin', (payload) => {
    if (payload.phase === 'batchDownload') {
        const batch = payload.batch;
        batch.on(event, listener);
    }
})

Note that you don't have to remove listeners yourself. They will be removed once the batch ends and is destroyed by the downloader.

Download Task

Each download task in a batch is represented by an object with the following properties:

Events

Each event emitted by a download task batch has a payload, which is an object with properties containing information about the event.

Changelog

v1.3.0

• Add support for multiple target URLs
• Add content.publishDate field to the content dir name format (PR #12 by kazuoteramoto)
• Bug fixes

v1.2.2

• Fix wrong file extension for some content types
• Fix YouTube API requests throwing errors due to YT changes

v1.2.1

• Bug fixes

v1.2.0

• Add support for granular control over:
  • posts to include in download based on type of media contained
  • the type of media to download
• Bug fixes

v1.1.1

• Fix initial data parsing following Patreon changes

v1.1.0

• Add support for downloading embedded YouTube videos

v1.0.1

• Fix missing types when importing as library
• Fix link in this README

v1.0.0

• Initial release