@dukanify/omnillm v1.0.14

OmniLLM SDK

A unified SDK for accessing multiple LLM providers (Ollama, OpenAI, Anthropic) through a single interface.

Installation

npm install @dukanify/omnillm

Usage

import { OmniLLM } from '@dukanify/omnillm';

// Initialize with configuration
const llm = new OmniLLM({
  baseURL: 'http://localhost:3000',     // Base URL for the proxy (without version)
  apiVersion: 'v1',                     // API version (defaults to 'v1')
  apiKey: 'ollama',                     // Default API key
  defaultModel: 'llama3.2:3b',          // Default model to use
  provider: 'ollama',                   // Default provider
  providerApiKey: 'your-api-key',       // Provider-specific API key
  fallback: {
    provider: 'openai',
    apiKey: process.env.OPENAI_API_KEY,
    model: 'gpt-3.5-turbo',
    embeddingModel: 'text-embedding-ada-002'
  },
  maxRetries: 3                         // Maximum number of retries
});

// Chat completion
const response = await llm.chat.completions.create({
  messages: [
    { role: 'user', content: 'Hello!' }
  ],
  model: 'llama2',
  temperature: 0.7,
  stream: false,
  tools: [], // Optional tools configuration
  tool_choice: 'auto', // Optional tool choice
  provider: 'ollama', // Optional provider override
  providerApiKey: 'your-api-key' // Optional provider API key override
});

// Embeddings
const embeddings = await llm.embeddings.create({
  input: 'Hello, world!',
  model: 'llama2',
  encoding_format: 'float', // or 'base64'
  provider: 'ollama', // Optional provider override
  providerApiKey: 'your-api-key' // Optional provider API key override
});

// List available models
const models = await llm.listModels();

// Check health status
const health = await llm.healthCheck();

Features

• Unified interface for multiple LLM providers (Ollama, OpenAI, Anthropic)
• Automatic fallback to OpenAI when the proxy is unavailable
• Support for chat completions and embeddings
• Automatic health checks for the proxy server
• Configurable retry mechanism
• TypeScript support
• Environment variable configuration support

Configuration

The SDK accepts the following configuration options:

interface OmniLLMConfig {
  baseURL?: string;           // Base URL for the proxy (default: http://localhost:3000)
  apiVersion?: string;        // API version (default: 'v1')
  apiKey?: string;            // API key for the proxy (default: 'ollama')
  defaultModel?: string;      // Default model to use (default: 'llama3.2:3b')
  provider?: Provider;        // Default provider ('ollama' | 'openai' | 'anthropic')
  providerApiKey?: string;    // Provider-specific API key
  fallback?: {
    provider: 'openai';
    apiKey: string;
    model?: string;
    embeddingModel?: string;  // Optional embedding model for fallback
  };
  maxRetries?: number;        // Maximum number of retries (default: 3)
}

Environment Variables

The SDK can be configured using environment variables:

• OLLAMA_PROXY_URL: Base URL for the proxy (without version)
• OLLAMA_DEFAULT_MODEL: Default model to use
• OLLAMA_PROVIDER: Default provider to use
• OLLAMA_PROVIDER_API_KEY: Provider-specific API key
• OLLAMA_PROXY_API_KEY: API key for the proxy

Fallback Behavior

When the proxy server is unavailable, the SDK will automatically fall back to the configured provider (OpenAI or Anthropic). This is useful for:

• Development environments where the proxy might not be running
• Production environments where you want a backup provider
• Testing different providers without changing code

Health Checks

The SDK performs automatic health checks on the proxy server every 30 seconds. You can also manually check the health status using the healthCheck() method, which returns:

{
  status: 'healthy' | 'unhealthy',
  details: {
    // ... proxy health details
  }
}

Examples

Check the examples directory for complete usage examples:

• full-demo.ts: Complete demonstration of all features
• fallback-test.ts: Testing fallback functionality

License

MIT