@panter/cloud-tasks NPM

🚀 Google Cloud Tasks, Simplified

A lightweight library that streamlines the use of Google Cloud Tasks.

🔹 Why Use This Library?

Using Google Cloud Tasks typically requires:

Implementing a worker service.
Defining and managing the API between the worker and client.
Handling task queue management and scheduling.
Setting up a local development environment.
Configuring Google Cloud Tasks to send HTTP requests to the worker.

This library eliminates much of this complexity by leveraging tRPC to define the API between the worker and the client.

✅ Key Features

Seamless API communication between workers and clients.
Simple task scheduling with an intuitive API.
Type safety and jump-to-definition support via tRPC.
Built-in support for local development with a Cloud Tasks emulator.

📂 Example Usage

Suppose we have one tasks worker (tasks-worker) and a backend API (api) that will be scheduling tasks.

Project Structure

An example project structure with Turborepo might look like this:

.
└── apps/
    ├── tasks-worker/
    │   ├── api.ts
    │   └── index.ts
    └── api/
        └── src/
            └── someEndpoint.ts

Pro tip: If you have multiple task workers, use naming convention tasks-worker-<name>.

Two apps that are deployed as two services: api and tasks-worker.

🛠 Step 1: Create the Task Worker (`api.ts`)

Define a tRPC-powered task server:

// apps/tasks-worker/api.ts

import { logger } from "@repo/logger";
// yarn add @panter/cloud-tasks
import { createTasksServer } from "@panter/cloud-tasks/server";
import { z } from "zod";

logger.info("Starting tasks server...");

export const { runServer, router } = createTasksServer((t) =>
  t.router({
    createUser: t.procedure
      .input(z.object({ name: z.string().min(5) }))
      .mutation(async (opts) => {
        logger.info(`creating user ${opts.input.name}`);
      }),
    doNothing: t.procedure.mutation(() => {
      logger.info("doing nothing");
    }),
    sendEmail: t.procedure.mutation(() => {
      logger.info("sending email");
    }),
  }),
);

// Export the router type for use in the client
export type Router = typeof router;

This creates a task server with three tRPC mutations: createUser, doNothing, and sendEmail.

Notice that mutations return nothing, as they are called by Google Cloud Tasks and their response is ignored.

🚀 Step 2: Start the Task Worker (`index.ts`)

Initialize the worker server:

// apps/tasks-worker/index.ts

import { logger } from "@repo/logger";
import { runServer } from "./api";

const port = parseInt(process.env.PORT);
runServer(port);
logger.info(`🚀 Task worker tRPC server running at http://localhost:${port}/`);

Pro tip: Set execution environment for tasks workers to "gen2", deny unauthenticated requests and set higher timeout in catladder:

// catladder.ts

"tasks-worker": {
  dir: "./apps/tasks-worker",
  deploy: {
    service: {
      allowUnauthenticated: false,
      executionEnvironment: "gen2",
      timeout: "1800s",
    },
  },
}

🏗 Step 3: Create the Task Client in `api`

Now, in the backend API (api), define a task client to send tasks:

// apps/api/src/someEndpoint.ts

import { builder } from "../builder";
import { logger } from "@repo/logger";

// NOTE: we import Router type from the tasks-worker without introducing a runtime dependency
import type { Router } from "../../tasks-worker-gitlab/api.ts";
// yarn add @panter/cloud-tasks
import { createTasksClient } from "@panter/cloud-tasks/client";

const tasks = createTasksClient<Router>({
  // TASKS_WORKER_URL is the URL of the task worker service
  tasksWorkerUrl: new URL(process.env.TASKS_WORKER_URL),
  // special characters are not allowed in queue name
  queueName: "reasonable-queue-name",
  // enable emulator in local development
  emulator: process.env.ENV_SHORT === "local" ? { port: 6020 } : false,
  // optional: provide a custom logger
  // assumes `logger.info(metadata, message)` order of arguments (pino logger)
  logger,
  // or provide custom adapter (e.g. for winston logger):
  // logger: {
  //   info: (meta, message) => logger.info(message, meta),
  //   warn: (meta, message) => logger.warn(message, meta),
  //   error: (meta, message) => logger.error(message, meta),
  // },
});

builder.mutationField("runJob", (t) =>
  t.field({
    type: "String",
    resolve: async () => {
      // Schedule a task with a payload
      await tasks.createUser.schedule({ name: "Bilbo Baggins" });
      //          ^^^^^^^^^^ Try jumping to definition here!
      return "ok";
    },
  }),
);

Pro tip: Set the TASKS_WORKER_URL environment variable to the URL of the task worker service in catladder:

// catladder.ts

api: {
  dir: "./apps/api",
  vars: {
    public: {
      TASKS_WORKER_URL: "${tasks-worker:ROOT_URL_INTERNAL}",
    }
  },
}

📌 Important: Avoid Circular Dependencies

Notice that Router is imported as a type and by relative path. This makes sure there won't be a circular dependency when e.g. taks-worker needs to import api to use some business logic.

🛠 Local Development: Using Cloud Tasks Emulator

For local testing, use the Cloud Tasks Emulator with Docker:

# docker-compose.yml

services:
  gcloud-tasks-emulator:
    image: ghcr.io/aertje/cloud-tasks-emulator:latest
    command: -host 0.0.0.0 -port 8123
    ports:
      - "6020:8123"
    # NOTE: Comment out `extra_hosts` if using Podman (see: https://github.com/containers/podman/issues/21681)
    extra_hosts:
      - "host.containers.internal:host-gateway"