horizon-eval-sdk v1.0.8

Horizon SDK

Node.js SDK for interacting with the Horizon Evaluations API.

Installation

npm install @horizon/sdk

Usage

import { HorizonClient } from '@horizon/sdk';

// Initialize the client with your API key
const client = new HorizonClient('your-api-key');

// Create an evaluation
const evaluation = await client.createEvaluation({
  name: 'Text Analysis Evaluation',
  description: 'Evaluates text analysis capabilities',
  taskFunction: `
    async function evaluate(input) {
      const text = input.text;
      return {
        length: text.length,
        wordCount: text.trim().split(/\s+/).length
      };
    }
  `,
  dataSnapshot: [
    {
      input: { text: "Hello world!" },
      expected: { length: 12, wordCount: 2 }
    }
  ],
  scoresConfig: [
    {
      name: "Length Accuracy",
      function: `
        async function score(input, output, expected) {
          return output.length === expected.length ? 10 : 0;
        }
      `,
      weight: 1
    }
  ]
});

// Create a run for an existing evaluation
const run = await client.createRun(evaluation.id, {
  taskFunction: `...`,
  dataSnapshot: [...],
  scoresConfig: [...]
});

// Get evaluation details
const evaluationDetails = await client.getEvaluation(evaluation.id);

// Get run details
const runDetails = await client.getRun(evaluation.id, run.id);

API Reference

HorizonClient

The main client for interacting with the Horizon API.

Constructor

new HorizonClient(apiKey: string, baseURL?: string)

• apiKey: Your Horizon API key
• baseURL: Optional base URL for the API (defaults to 'http://localhost:3000/api/horizon')

Methods

createEvaluation(config: EvaluationConfig)

Creates a new evaluation.

interface EvaluationConfig {
  name: string;
  description?: string;
  taskFunction?: string;
  dataSnapshot?: any[];
  scoresConfig?: Array<{
    name: string;
    function: string;
    weight: number;
  }>;
}

createRun(evaluationId: string, config: RunConfig)

Creates a new run for an existing evaluation.

interface RunConfig {
  taskFunction: string;
  dataSnapshot: any[];
  scoresConfig: Array<{
    name: string;
    function: string;
    weight: number;
  }>;
}

getEvaluation(evaluationId: string)

Gets details of an evaluation including its runs.

getRun(evaluationId: string, runId: string)

Gets details of a specific run including its results.

Error Handling

The SDK throws errors for:

• Invalid API key
• Network issues
• API errors (4xx, 5xx responses)

Example error handling:

try {
  const evaluation = await client.createEvaluation({...});
} catch (error) {
  if (error.response) {
    // The request was made and the server responded with an error
    console.error('API error:', error.response.data);
  } else if (error.request) {
    // The request was made but no response was received
    console.error('Network error:', error.request);
  } else {
    // Something happened in setting up the request
    console.error('Error:', error.message);
  }
}