@kluai/gateway v1.1.2

Klu Gateway

Klu gateway is an OpenAI compatible gateway package. With it you can self host your own OpenAI API that supports the following models:

• OpenAI
• Azure OpenAI
• Perplexity
• Anthropic
• Replicate
• Google (Gemini models)

Installation

To install the gateway simply run the following command:

npm i @kluai/gateway

We support node versions 14+. If you are using an older version of node, please upgrade.

Running on edge

In order to run on edge (Cloudflare for example) you need to copy the cloudflare directory in this repository.

Once you've copied over the code into your own project, create a cloud worker and run the following command:

npm i
wrangler dev

Now you will be able to run this locally. Feel free to tweak the implementation as much as you need and once you are ready to deploy run:

wrangler deploy

Running on cloud

If you'd like to run this into a cloud you can import the OpenAI gateway function like so.

import { OpenAI } from "@kluai/gateway";

export default {
  async fetch(request: Request, context: ExecutionContext) {
    return OpenAI()(request, context);
  },
};

You could run this in a lambda or any other server and you would be ready to handle any number of requests.

For feedback, suggestions or problems feel free to email stefan@klu.ai