@crossingminds/ragsys-sdk v0.2.2

RAGSys SDK

This library provides tools for using Crossing Minds's RAGSys AI Retrieval Engine. See the API documentation for more information on RAGSys usage.

Installation

npm install @crossingminds/ragsys-sdk

# or

pnpm add @crossingminds/ragsys-sdk

Usage

import {
  createChatCompletion,
  initializeNewDatabase,
  GENERIC_RAGSYS_SYSTEM_PROMPT,
  type Model,
} from "@crossingminds/ragsys-sdk";

(async () => {
  /*
    In order to do retrieval or LLM inference, we need to create a RAGSys database.
    Databases persist in the RAGSys service, so you'll only need to do this once per set of few-shot examples.
    */
  const databaseName = "capital-letters";

  /*
    The SDK provides low-level tools to call individual API endpoints, but it also provides high-level tools to make common workflows easier.

    Here we can use the `initializeNewDatabase` tool to create a new database, add examples to it, and train a retrieval model.
    */
  const initializationResult = await initializeNewDatabase({
    apiKey: YOUR_RAGSYS_API_KEY,
    databaseName,
    description: "A trivial demonstration of capitalizing a single letter",
    /*
      These few-shot examples can be provided to LLMs as ICL ("in context learning").
      `query` is usually an example of a user message, and `response` is the expected LLM response.
      */
    examples: [
      {
        query: "a",
        response: "A",
      },
      {
        query: "b",
        response: "B",
      },
      {
        query: "c",
        response: "C",
      },
    ],
    onStatusChange: (initializationStatus) => {
      console.log("initializationStatus: ", initializationStatus);
    },
  });

  if (initializationResult.error) {
    console.error(initializationResult.error);
    throw Error("Failed to initialize a new database");
  }

  /*
    Our new RAGSys database has been successfully setup!
    We're ready to retrieve few-shot examples and generate LLM responses.
    */

  /*
    "chat completion" means we're going to ask an LLM chat model to generate content.
    We'll choose to provide it with few-shot examples so it understands how we want it to format its response.
    */
  const ragsysResult = await createChatCompletion({
    apiKey: YOUR_RAGSYS_API_KEY,
    databaseName,
    /* There are about 100 different LLMs to choose from. */
    model: "openai/gpt-4o-mini" satisfies Model,
    messages: [
      /*
        The system prompt is where the few-shot examples are provided to the LLM.
        You can provide your own prompt, or use the generic prompt provided by the RAGSys SDK.
        */
      { role: "system", content: GENERIC_RAGSYS_SYSTEM_PROMPT },
      /*
        Here the user message resembles the `query` properties of our few-shot examples.
        The LLM should easily be able to understand the response format we're interested in.
        */
      { role: "user", content: "z" },
    ],
    /*
      By specifying `retrieval` options, we're using RAGSys
      to retrieve the best few-shot examples for the given user message.
      If `retrieval` is not specified, the LLM will generate a completion without any few-shot examples from the database.
      */
    retrieval: {
      amt: 3,
    },
  });

  if (ragsysResult.error) {
    console.error(ragsysResult.error);
    throw Error("Failed to generate a chat completion");
  }

  const generatedMessage = ragsysResult.data.choices[0]?.message.content;

  console.log(generatedMessage); // Output: "Z"

  /* That's it! We've done our first chat completion using RAGSys. */
})();

Library Exports

createChatCompletion

Generate content using chat LLM models with optional RAGSys retrieval.

https://ragsys-api.crossingminds.com/redoc#tag/Completion/operation/completion

createDatabase

Creates a new RAGSys database.

https://ragsys-api.crossingminds.com/redoc#tag/Database/operation/create_database

createOrUpdateExamples

Create or update few-shot examples in a RAGSys database. After creating examples, run trainRetrievalModel if this is the first time few-shot examples have been added to this database.

https://ragsys-api.crossingminds.com/redoc#tag/Few-Shot-Example/operation/save_examples

getDatabase

Get basic information about a RAGSys database.

https://ragsys-api.crossingminds.com/redoc#tag/Database/operation/get_database

getDatabases or getDatabaseIteratively

Get a list of RAGSys databases.

https://ragsys-api.crossingminds.com/redoc#tag/Database/operation/get_databases

getExamples or getExamplesIteratively

Get few-shot examples stored in a RAGSys database.

https://ragsys-api.crossingminds.com/redoc#tag/Few-Shot-Example/operation/get_saved_examples

getModels

Get a list of LLMs ("Large Language Models") available for use with chat completion.

https://ragsys-api.crossingminds.com/redoc#tag/Completion/operation/get_models

getTask

Get the status for a RAGSys service task. Use the higher-level waitForTaskToSettle to repeatedly poll the API using this function until the task is complete.

https://ragsys-api.crossingminds.com/redoc#tag/Task/operation/get_task

retrieveRankedExamples

Retrieve few-shot examples ranked by relevance based on the given query. This function runs the core RAGSys retrieval algorithm without calling an LLM.

https://ragsys-api.crossingminds.com/redoc#tag/Few-Shot-Example/operation/retrieve_ranked_examples

trainRetrievalModel

Train a the retrieval model for a new RAGSys database. This function only needs to be run once per database: After the first run, the retrieval model will be updated automatically as new data is added. A database must have few-shot examples before training the retrieval model. You can wait for the model to be trained using the waitForTaskToSettle method returned by this function.

https://ragsys-api.crossingminds.com/redoc#tag/Task/operation/train_retrieval_model

initializeNewDatabase

Setup a new RAGSys database with examples. Combines the createDatabase, createOrUpdateExamples, and trainRetrievalModel commands. This may take several minutes depending on the size and quantity of your examples.

waitForTaskToSettle

Polls the getTask API until the specified task is complete. You may not need to use this function directly as it is typically part of the return object of any function that creates a task; including .trainRetrievalModel and createOrUpdateExamples.

GENERIC_RAGSYS_SYSTEM_PROMPT

A generic system prompt for use with RAGSys retrieval. Includes Jinja template for inserting few-shot examples.

STANDARD_FEW_SHOT_EXAMPLE_TEMPLATE

Standard Jinja template for inserting formatted few-shot examples in a system prompt.

type Model

Names of LLMs available for use by createChatCompletion. For the most up-to-date list, use the getModels function to get the model names as an array of strings.