replicate v0.29.4

Replicate Node.js client

A Node.js client for Replicate. It lets you run models from your Node.js code, and everything else you can do with Replicate's HTTP API.

!IMPORTANT This library can't interact with Replicate's API directly from a browser. For more information about how to build a web application check out our "Build a website with Next.js" guide.

Installation

Install it from npm:

npm install replicate

Usage

Create the client:

// CommonJS (default or using .cjs extension)
const Replicate = require("replicate");

// ESM (where `"module": true` in package.json or using .mjs extension)
import Replicate from "replicate";

const replicate = new Replicate({
  // get your token from https://replicate.com/account
  auth: "my api token", // defaults to process.env.REPLICATE_API_TOKEN
});

Run a model and await the result:

const model = "stability-ai/stable-diffusion:27b93a2413e7f36cd83da926f3656280b2931564ff050bf9575f1fdf9bcd7478";
const input = {
  prompt: "a 19th century portrait of a raccoon gentleman wearing a suit",
};
const output = await replicate.run(model, { input });
// ['https://replicate.delivery/pbxt/GtQb3Sgve42ZZyVnt8xjquFk9EX5LP0fF68NTIWlgBMUpguQA/out-0.png']

You can also run a model in the background:

let prediction = await replicate.predictions.create({
  version: "27b93a2413e7f36cd83da926f3656280b2931564ff050bf9575f1fdf9bcd7478",
  input: {
    prompt: "painting of a cat by andy warhol",
  },
});

Then fetch the prediction result later:

prediction = await replicate.predictions.get(prediction.id);

Or wait for the prediction to finish:

prediction = await replicate.wait(prediction);
console.log(prediction.output);
// ['https://replicate.delivery/pbxt/RoaxeXqhL0xaYyLm6w3bpGwF5RaNBjADukfFnMbhOyeoWBdhA/out-0.png']

To run a model that takes a file input you can pass the data directly or pass a URL to a publicly accessible file.

const fs = require("node:fs/promises");

// Or when using ESM.
// import fs from "node:fs/promises";

const model = "nightmareai/real-esrgan:42fed1c4974146d4d2414e2be2c5277c7fcf05fcc3a73abf41610695738c1d7b";
const input = {
  image: await fs.readFile("path/to/image.png"),
};
const output = await replicate.run(model, { input });
// ['https://replicate.delivery/mgxm/e7b0e122-9daa-410e-8cde-006c7308ff4d/output.png']

Webhooks

Webhooks provide real-time updates about your prediction. Specify an endpoint when you create a prediction, and Replicate will send HTTP POST requests to that URL when the prediction is created, updated, and finished.

It is possible to provide a URL to the predictions.create() function that will be requested by Replicate when the prediction status changes. This is an alternative to polling.

To receive webhooks you’ll need a web server. The following example uses Hono, a web standards based server, but this pattern applies to most frameworks.

import { serve } from '@hono/node-server';
import { Hono } from 'hono';

const app = new Hono();
app.get('/webhooks/replicate', async (c) => {
  // Get the prediction from the request.
  const prediction = await c.req.json();
  console.log(prediction);
  //=> {"id": "xyz", "status": "successful", ... }

  // Acknowledge the webhook.
  c.status(200);
  c.json({ok: true});
}));

serve(app, (info) => {
  console.log(`Listening on http://localhost:${info.port}`)
  //=> Listening on http://localhost:3000
});

Create the prediction passing in the webhook URL to webhook and specify which events you want to receive in webhook_events_filter out of "start", "output", ”logs” and "completed":

const Replicate = require("replicate");
const replicate = new Replicate();

const input = {
    image: "https://replicate.delivery/pbxt/KWDkejqLfER3jrroDTUsSvBWFaHtapPxfg4xxZIqYmfh3zXm/Screenshot%202024-02-28%20at%2022.14.00.png",
    denoising_strength: 0.5,
    instant_id_strength: 0.8
};

const callbackURL = `https://my.app/webhooks/replicate`;
await replicate.predictions.create({
  version: "19deaef633fd44776c82edf39fd60e95a7250b8ececf11a725229dc75a81f9ca",
  input: input,
  webhook: callbackURL,
  webhook_events_filter: ["completed"],
});

// The server will now handle the event and log:
// => {"id": "xyz", "status": "successful", ... }

TypeScript

Currently in order to support the module format used by replicate you'll need to set esModuleInterop to true in your tsconfig.json.

API

Constructor

const replicate = new Replicate(options);

The client makes requests to Replicate's API using fetch. By default, the globalThis.fetch function is used, which is available on Node.js 18 and later, as well as Cloudflare Workers, Vercel Edge Functions, and other environments.

On earlier versions of Node.js and other environments where global fetch isn't available, you can install a fetch function from an external package like cross-fetch and pass it to the fetch option in the constructor.

const Replicate = require("replicate");
const fetch = require("fetch");

// Using ESM:
// import Replicate from "replicate";
// import fetch from "cross-fetch";

const replicate = new Replicate({ fetch });

You can also use the fetch option to add custom behavior to client requests, such as injecting headers or adding log statements.

const customFetch = (url, options) => {
  const headers = options && options.headers ? { ...options.headers } : {};
  headers["X-Custom-Header"] = "some value";

  console.log("fetch", { url, ...options, headers });

  return fetch(url, { ...options, headers });
};

const replicate = new Replicate({ fetch: customFetch });

replicate.run

Run a model and await the result. Unlike replicate.prediction.create, this method returns only the prediction output rather than the entire prediction object.

const output = await replicate.run(identifier, options, progress);

Throws Error if the prediction failed.

Returns Promise<object> which resolves with the output of running the model.

Example:

const model = "stability-ai/sdxl:8beff3369e81422112d93b89ca01426147de542cd4684c244b673b105188fe5f";
const input = { prompt: "a 19th century portrait of a raccoon gentleman wearing a suit" };
const output = await replicate.run(model, { input });

Example that logs progress as the model is running:

const model = "stability-ai/sdxl:8beff3369e81422112d93b89ca01426147de542cd4684c244b673b105188fe5f";
const input = { prompt: "a 19th century portrait of a raccoon gentleman wearing a suit" };
const onProgress = (prediction) => {
   const last_log_line = prediction.logs.split("\n").pop()
   console.log({id: prediction.id, log: last_log_line})
}
const output = await replicate.run(model, { input }, onProgress)

replicate.stream

Run a model and stream its output. Unlike replicate.prediction.create, this method returns only the prediction output rather than the entire prediction object.

for await (const event of replicate.stream(identifier, options)) { /* ... */ }

Throws Error if the prediction failed.

Returns AsyncGenerator<ServerSentEvent> which yields the events of running the model.

Example:

const model = "meta/llama-2-70b-chat";
const options = {
  input: {
    prompt: "Write a poem about machine learning in the style of Mary Oliver.",
  },
  // webhook: "https://smee.io/dMUlmOMkzeyRGjW" // optional
};
const output = [];

for await (const { event, data } of replicate.stream(model, options)) {
  if (event === "output") {
    output.push(data);
  }
}

console.log(output.join("").trim());

Server-sent events

A stream generates server-sent events with the following properties:

As the prediction runs, the generator yields output and logs events. If an error occurs, the generator yields an error event with a JSON object containing the error message set to the data property. When the prediction is done, the generator yields a done event with an empty JSON object set to the data property.

Events with the output event type have their toString() method overridden to return the event data as a string. Other event types return an empty string.

replicate.models.get

Get metadata for a public model or a private model that you own.

const response = await replicate.models.get(model_owner, model_name);

{
  "url": "https://replicate.com/replicate/hello-world",
  "owner": "replicate",
  "name": "hello-world",
  "description": "A tiny model that says hello",
  "visibility": "public",
  "github_url": "https://github.com/replicate/cog-examples",
  "paper_url": null,
  "license_url": null,
  "latest_version": {
    /* ... */
  }
}

replicate.models.list

Get a paginated list of all public models.

const response = await replicate.models.list();

{
  "next": null,
  "previous": null,
  "results": [
    {
      "url": "https://replicate.com/replicate/hello-world",
      "owner": "replicate",
      "name": "hello-world",
      "description": "A tiny model that says hello",
      "visibility": "public",
      "github_url": "https://github.com/replicate/cog-examples",
      "paper_url": null,
      "license_url": null,
      "run_count": 5681081,
      "cover_image_url": "...",
      "default_example": {
        /* ... */
      },
      "latest_version": {
        /* ... */
      }
    }
  ]
}

replicate.models.create

Create a new public or private model.

const response = await replicate.models.create(model_owner, model_name, options);

replicate.hardware.list

List available hardware for running models on Replicate.

const response = await replicate.hardware.list()

[
  {"name": "CPU", "sku": "cpu" },
  {"name": "Nvidia T4 GPU", "sku": "gpu-t4" },
  {"name": "Nvidia A40 GPU", "sku": "gpu-a40-small" },
  {"name": "Nvidia A40 (Large) GPU", "sku": "gpu-a40-large" },
]

replicate.models.versions.list

Get a list of all published versions of a model, including input and output schemas for each version.

const response = await replicate.models.versions.list(model_owner, model_name);

{
  "previous": null,
  "next": null,
  "results": [
    {
      "id": "5c7d5dc6dd8bf75c1acaa8565735e7986bc5b66206b55cca93cb72c9bf15ccaa",
      "created_at": "2022-04-26T19:29:04.418669Z",
      "cog_version": "0.3.0",
      "openapi_schema": {
        /* ... */
      }
    },
    {
      "id": "e2e8c39e0f77177381177ba8c4025421ec2d7e7d3c389a9b3d364f8de560024f",
      "created_at": "2022-03-21T13:01:04.418669Z",
      "cog_version": "0.3.0",
      "openapi_schema": {
        /* ... */
      }
    }
  ]
}

replicate.models.versions.get

Get metatadata for a specific version of a model.

const response = await replicate.models.versions.get(model_owner, model_name, version_id);

{
  "id": "5c7d5dc6dd8bf75c1acaa8565735e7986bc5b66206b55cca93cb72c9bf15ccaa",
  "created_at": "2022-04-26T19:29:04.418669Z",
  "cog_version": "0.3.0",
  "openapi_schema": {
    /* ... */
  }
}

replicate.collections.get

Get a list of curated model collections. See replicate.com/collections.

const response = await replicate.collections.get(collection_slug);

replicate.predictions.create

Run a model with inputs you provide.

const response = await replicate.predictions.create(options);

{
  "id": "ufawqhfynnddngldkgtslldrkq",
  "version": "5c7d5dc6dd8bf75c1acaa8565735e7986bc5b66206b55cca93cb72c9bf15ccaa",
  "status": "succeeded",
  "input": {
    "text": "Alice"
  },
  "output": null,
  "error": null,
  "logs": null,
  "metrics": {},
  "created_at": "2022-04-26T22:13:06.224088Z",
  "started_at": null,
  "completed_at": null,
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/ufawqhfynnddngldkgtslldrkq",
    "cancel": "https://api.replicate.com/v1/predictions/ufawqhfynnddngldkgtslldrkq/cancel",
    "stream": "https://streaming.api.replicate.com/v1/predictions/ufawqhfynnddngldkgtslldrkq" // Present only if `options.stream` is `true`
  }
}

Streaming

Specify the stream option when creating a prediction to request a URL to receive streaming output using server-sent events (SSE).

If the requested model version supports streaming, then the returned prediction will have a stream entry in its urls property with a URL that you can use to construct an EventSource.

if (prediction && prediction.urls && prediction.urls.stream) {
  const source = new EventSource(prediction.urls.stream, { withCredentials: true });

  source.addEventListener("output", (e) => {
    console.log("output", e.data);
  });

  source.addEventListener("error", (e) => {
    console.error("error", JSON.parse(e.data));
  });

  source.addEventListener("done", (e) => {
    source.close();
    console.log("done", JSON.parse(e.data));
  });
}

A prediction's event stream consists of the following event types:

A done event is emitted when a prediction finishes successfully, is cancelled, or produces an error.

replicate.predictions.get

const response = await replicate.predictions.get(prediction_id);

{
  "id": "ufawqhfynnddngldkgtslldrkq",
  "version": "5c7d5dc6dd8bf75c1acaa8565735e7986bc5b66206b55cca93cb72c9bf15ccaa",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/ufawqhfynnddngldkgtslldrkq",
    "cancel": "https://api.replicate.com/v1/predictions/ufawqhfynnddngldkgtslldrkq/cancel"
  },
  "status": "starting",
  "input": {
    "text": "Alice"
  },
  "output": null,
  "error": null,
  "logs": null,
  "metrics": {},
  "created_at": "2022-04-26T22:13:06.224088Z",
  "started_at": null,
  "completed_at": null
}

replicate.predictions.cancel

Stop a running prediction before it finishes.

const response = await replicate.predictions.cancel(prediction_id);

{
  "id": "ufawqhfynnddngldkgtslldrkq",
  "version": "5c7d5dc6dd8bf75c1acaa8565735e7986bc5b66206b55cca93cb72c9bf15ccaa",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/ufawqhfynnddngldkgtslldrkq",
    "cancel": "https://api.replicate.com/v1/predictions/ufawqhfynnddngldkgtslldrkq/cancel"
  },
  "status": "canceled",
  "input": {
    "text": "Alice"
  },
  "output": null,
  "error": null,
  "logs": null,
  "metrics": {},
  "created_at": "2022-04-26T22:13:06.224088Z",
  "started_at": "2022-04-26T22:13:06.224088Z",
  "completed_at": "2022-04-26T22:13:06.224088Z"
}

replicate.predictions.list

Get a paginated list of all the predictions you've created.

const response = await replicate.predictions.list();

replicate.predictions.list() takes no arguments.

{
  "previous": null,
  "next": "https://api.replicate.com/v1/predictions?cursor=cD0yMDIyLTAxLTIxKzIzJTNBMTglM0EyNC41MzAzNTclMkIwMCUzQTAw",
  "results": [
    {
      "id": "jpzd7hm5gfcapbfyt4mqytarku",
      "version": "b21cbe271e65c1718f2999b038c18b45e21e4fba961181fbfae9342fc53b9e05",
      "urls": {
        "get": "https://api.replicate.com/v1/predictions/jpzd7hm5gfcapbfyt4mqytarku",
        "cancel": "https://api.replicate.com/v1/predictions/jpzd7hm5gfcapbfyt4mqytarku/cancel"
      },
      "source": "web",
      "status": "succeeded",
      "created_at": "2022-04-26T20:00:40.658234Z",
      "started_at": "2022-04-26T20:00:84.583803Z",
      "completed_at": "2022-04-26T20:02:27.648305Z"
    }
    /* ... */
  ]
}

replicate.trainings.create

Use the training API to fine-tune language models to make them better at a particular task. To see what language models currently support fine-tuning, check out Replicate's collection of trainable language models.

If you're looking to fine-tune image models, check out Replicate's guide to fine-tuning image models.

const response = await replicate.trainings.create(model_owner, model_name, version_id, options);

{
  "id": "zz4ibbonubfz7carwiefibzgga",
  "version": "3ae0799123a1fe11f8c89fd99632f843fc5f7a761630160521c4253149754523",
  "status": "starting",
  "input": {
    "text": "..."
  },
  "output": null,
  "error": null,
  "logs": null,
  "started_at": null,
  "created_at": "2023-03-28T21:47:58.566434Z",
  "completed_at": null
}

Warning If you try to fine-tune a model that doesn't support training, you'll get a 400 Bad Request response from the server.

replicate.trainings.get

Get metadata and status of a training.

const response = await replicate.trainings.get(training_id);

{
  "id": "zz4ibbonubfz7carwiefibzgga",
  "version": "3ae0799123a1fe11f8c89fd99632f843fc5f7a761630160521c4253149754523",
  "status": "succeeded",
  "input": {
    "data": "..."
    "param1": "..."
  },
  "output": {
    "version": "..."
  },
  "error": null,
  "logs": null,
  "webhook_completed": null,
  "started_at": "2023-03-28T21:48:02.402755Z",
  "created_at": "2023-03-28T21:47:58.566434Z",
  "completed_at": "2023-03-28T02:49:48.492023Z"
}

replicate.trainings.cancel

Stop a running training job before it finishes.

const response = await replicate.trainings.cancel(training_id);

{
  "id": "zz4ibbonubfz7carwiefibzgga",
  "version": "3ae0799123a1fe11f8c89fd99632f843fc5f7a761630160521c4253149754523",
  "status": "canceled",
  "input": {
    "data": "..."
    "param1": "..."
  },
  "output": {
    "version": "..."
  },
  "error": null,
  "logs": null,
  "webhook_completed": null,
  "started_at": "2023-03-28T21:48:02.402755Z",
  "created_at": "2023-03-28T21:47:58.566434Z",
  "completed_at": "2023-03-28T02:49:48.492023Z"
}

replicate.trainings.list

Get a paginated list of all the trainings you've run.

const response = await replicate.trainings.list();

replicate.trainings.list() takes no arguments.

{
  "previous": null,
  "next": "https://api.replicate.com/v1/trainings?cursor=cD0yMDIyLTAxLTIxKzIzJTNBMTglM0EyNC41MzAzNTclMkIwMCUzQTAw",
  "results": [
    {
      "id": "jpzd7hm5gfcapbfyt4mqytarku",
      "version": "b21cbe271e65c1718f2999b038c18b45e21e4fba961181fbfae9342fc53b9e05",
      "urls": {
        "get": "https://api.replicate.com/v1/trainings/jpzd7hm5gfcapbfyt4mqytarku",
        "cancel": "https://api.replicate.com/v1/trainings/jpzd7hm5gfcapbfyt4mqytarku/cancel"
      },
      "source": "web",
      "status": "succeeded",
      "created_at": "2022-04-26T20:00:40.658234Z",
      "started_at": "2022-04-26T20:00:84.583803Z",
      "completed_at": "2022-04-26T20:02:27.648305Z"
    }
    /* ... */
  ]
}

replicate.deployments.predictions.create

Run a model using your own custom deployment.

Deployments allow you to run a model with a private, fixed API endpoint. You can configure the version of the model, the hardware it runs on, and how it scales. See the deployments guide to learn more and get started.

const response = await replicate.deployments.predictions.create(deployment_owner, deployment_name, options);

Use replicate.wait to wait for a prediction to finish, or replicate.predictions.cancel to cancel a prediction before it finishes.

replicate.deployments.list

List your deployments.

const response = await replicate.deployments.list();

{
  "next": null,
  "previous": null,
  "results": [
    {
      "owner": "acme",
      "name": "my-app-image-generator",
      "current_release": { /* ... */ }
    }
    /* ... */
  ]
}

replicate.deployments.create

Create a new deployment.

const response = await replicate.deployments.create(options);

{
  "owner": "acme",
  "name": "my-app-image-generator",
  "current_release": {
    "number": 1,
    "model": "stability-ai/sdxl",
    "version": "da77bc59ee60423279fd632efb4795ab731d9e3ca9705ef3341091fb989b7eaf",
    "created_at": "2024-03-14T11:43:32.049157Z",
    "created_by": {
       "type": "organization",
       "username": "acme",
       "name": "Acme, Inc.",
       "github_url": "https://github.com/replicate"
    },
    "configuration": {
      "hardware": "gpu-a100",
      "min_instances": 1,
      "max_instances": 0
    }
  }
}

replicate.deployments.update

Update an existing deployment.

const response = await replicate.deployments.update(deploymentOwner, deploymentName, options);

{
  "owner": "acme",
  "name": "my-app-image-generator",
  "current_release": {
    "number": 2,
    "model": "stability-ai/sdxl",
    "version": "39ed52f2a78e934b3ba6e2a89f5b1c712de7dfea535525255b1aa35c5565e08b",
    "created_at": "2024-03-14T11:43:32.049157Z",
    "created_by": {
       "type": "organization",
       "username": "acme",
       "name": "Acme, Inc.",
       "github_url": "https://github.com/replicate"
    },
    "configuration": {
      "hardware": "gpu-a100",
      "min_instances": 1,
      "max_instances": 0
    }
  }
}

replicate.paginate

Pass another method as an argument to iterate over results that are spread across multiple pages.

This method is implemented as an async generator function, which you can use in a for loop or iterate over manually.

// iterate over paginated results in a for loop
for await (const page of replicate.paginate(replicate.predictions.list)) {
  /* do something with page of results */
}

// iterate over paginated results one at a time
let paginator = replicate.paginate(replicate.predictions.list);
const page1 = await paginator.next();
const page2 = await paginator.next();
// etc.

replicate.request

Low-level method used by the Replicate client to interact with API endpoints.

const response = await replicate.request(route, parameters);

The replicate.request() method is used by the other methods to interact with the Replicate API. You can call this method directly to make other requests to the API.

TypeScript

The Replicate constructor and all replicate.* methods are fully typed.

Vendored Dependencies

We have a few dependencies that have been bundled into the vendor directory rather than adding external npm dependencies.

These have been generated using bundlejs.com and copied into the appropriate directory along with the license and repository information.

• eventsource-parser/stream
• streams-text-encoding/text-decoder-stream

!NOTE The vendored implementation of TextDecoderStream requires the following patch to be applied to the output of bundlejs.com:

  constructor(label, options) {
-   this[decDecoder] = new TextDecoder(label, options);
-   this[decTransform] = new TransformStream(new TextDecodeTransformer(this[decDecoder]));
+   const decoder = new TextDecoder(label || "utf-8", options || {});
+   this[decDecoder] = decoder;
+   this[decTransform] = new TransformStream(new TextDecodeTransformer(decoder));
  }