@eu-ge-ne/puppeteer-pool v2.0.1

@eu-ge-ne/puppeteer-pool

Page pooling for Puppeteer. Written in TypeScript. Each page is opened in new browser instance.

Install | Example | API | License

Install

$ npm install @eu-ge-ne/puppeteer-pool

Example

import { PuppeteerPool } from "@eu-ge-ne/puppeteer-pool";

const pool = new PuppeteerPool({
    concurrency: 100,
    launchOptions: {
        headless: true,
    },
});

let page!: Page;

try {
    page = await pool.acquire();
    page.on("error", err => console.log(err));
} finally {
    if (page) {
        pool.close(page);
    }
}

API

Create instance

import { PuppeteerPool, LaunchOptions } from "@eu-ge-ne/puppeteer-pool";

/**
 * Concurrency of the pool
 * Defaults to 1
 */
const concurrency = 100;

/**
 * Maximum time in milliseconds to wait for acquire
 * Defaults to 30_000
 * Should be equal or greater than launchOptions.timeout
 */
const acquireTimeout: number = 60_000;

/**
 * Options, provided to default puppeteer.launch()
 */
const launchOptions: LaunchOptions = {};

const pool = new PuppeteerPool({ concurrency, acquireTimeout, launchOptions });

Or, provide launch option with function () => Promise<Browser>:

import { PuppeteerPool, launch } from "@eu-ge-ne/puppeteer-pool";

const pool = new PuppeteerPool({
    concurrency: 100,
    launch: () => launch({
        headless: true,
    }),
});

Acquire page

const page = await pool.acquire();

Close page

pool.close(page);

Close all pages

pool.closeAll();

Get stats

const stats = pool.stats();

const {
    // array of browser descriptors
    browsers: [
        {
            // browser instance lifetime in ms
            // (instance will be destroyed when `counter === concurrency` and `active === 0`)
            lifetime,
            // number of pages, opened by this browser instance
            // (max value is equal to `concurrency` parameter)
            counter,
            // number of active pages
            active,
        },
        ...rest
    ],
    acquireTime: {
        // max acquire time
        max,
        // mean acquire time
        mean,
    }
} = stats;

Events

after_acquire

Emitted after page acquired. Useful for additional page setup:

const pool = new PuppeteerPool<{ attempt: number }>({ concurrency: 100 });

pool.on("after_acquire", (page, opts) => {
    const timeout = opts.attempt * 1000 * 60;

    page.setDefaultNavigationTimeout(timeout);
    page.setDefaultTimeout(timeout);
});

// ...

const page = await pool.acquire({ attempt });

page.on("error", err => console.log(err));
page.on("console", msg => console.log(msg));

after_close

Emitted after page closed. For example can be used for unsubscribing:

// ...

pool.on("after_close", page => page.removeAllListeners());

error

Emitted when error occurred

// ...

pool.on("error", err => console.log(err));

License

MIT