# vellma

> Virtual, Eloquent LLM Assistants

Latest version **0.8.1** (published 2024-05-19) · MIT license · 0 weekly downloads

## Install

```sh
npm install vellma
pnpm add vellma
yarn add vellma
bun add vellma
```

## Health

**Score 30/100 (F)** — status: abandoned.

Positive: has types; esm support; no vulnerabilities; high quality score.

Warnings: low downloads; pre 1.0.

Negative: abandoned; low maintenance score.

## Facts

| | |
|---|---|
| Version | 0.8.1 |
| Published | 2024-05-19 |
| First published | 2023-07-09 |
| Weekly downloads | 0 |
| License | MIT |
| TypeScript types | bundled |
| Module format | ESM + CommonJS |
| Dependencies | 8 |
| Unpacked size | 551.5 KB |
| Known vulnerabilities | 0 (+18 in 3 direct dependencies) |
| Install scripts | no |
| GitHub stars | 7 |
| Author | David Myers |
| Maintainers | voraciousdev, davidmyersdev |
| Keywords | chat, chatgpt, gpt, langchain, language-models, llm, openai, terminal, tty |

## Links

- npm: https://www.npmjs.com/package/vellma
- Repository: https://github.com/davidmyersdev/vellma
- Issues: https://github.com/davidmyersdev/vellma/issues
- Funding: https://github.com/sponsors/davidmyersdev
- npm.io page: https://npm.io/package/vellma

## Dependencies (8)

- [zod](https://npm.io/package/zod.md) ^3.21.4
- [chalk](https://npm.io/package/chalk.md) ^5.2.0
- [events](https://npm.io/package/events.md) ^3.3.0
- [nanoid](https://npm.io/package/nanoid.md) ^4.0.2
- [openai](https://npm.io/package/openai.md) ^3.3.0
- [undici](https://npm.io/package/undici.md) ^5.23.0
- [langchain](https://npm.io/package/langchain.md) ^0.0.102
- [ml-distance](https://npm.io/package/ml-distance.md) ^4.0.1

## Alternatives

- [localforage](https://npm.io/package/localforage.md) — 6.2M weekly downloads
- [localforage-observable](https://npm.io/package/localforage-observable.md) — 30.8K weekly downloads
- [@y/y](https://npm.io/package/@y/y.md) — 30.1K weekly downloads
- [@metaobjectsdev/render](https://npm.io/package/@metaobjectsdev/render.md) — 3.5K weekly downloads
- [@ledgerhq/coin-algorand](https://npm.io/package/@ledgerhq/coin-algorand.md) — 1.1K weekly downloads

## Recent versions

- 0.8.1 (latest) — 2024-05-19
- 0.8.0 — 2024-05-15
- 0.7.0 — 2023-09-02
- 0.6.0 — 2023-08-23
- 0.5.0 — 2023-08-16
- 0.4.0 — 2023-07-19
- 0.3.0 — 2023-07-09

## README

[![Sponsor me](https://img.shields.io/badge/sponsor-DB61A2?style=for-the-badge&logo=GitHub-Sponsors&logoColor=white)](https://voracious.link/sponsor)
[![Donate](https://img.shields.io/badge/donate-FF5F5F?style=for-the-badge&logo=ko-fi&logoColor=white)](https://voracious.link/donate)

# `vellma` <!-- omit in toc -->

Virtual, Eloquent LLM Assistants

## Overview <!-- omit in toc -->

- [How to use `vellma`](#how-to-use-vellma)
  - [Example: Terminal chat](#example-terminal-chat)
- [Concepts](#concepts)
  - [Interfaces and adapters](#interfaces-and-adapters)
    - [Example: Mapping `api.openai.com/v1/chat/completions` to `ChatIntegration`](#example-mapping-apiopenaicomv1chatcompletions-to-chatintegration)
  - [Peripherals](#peripherals)
    - [Example: Getting input from or displaying output to a user](#example-getting-input-from-or-displaying-output-to-a-user)
  - [Models](#models)
    - [Embedding models](#embedding-models)
  - [Integrations](#integrations)
- [How to contribute to `vellma`](#how-to-contribute-to-vellma)
  - [Set up your development environment](#set-up-your-development-environment)

## How to use `vellma`

Install it with your preferred package manager.

```bash
# npm
npm i vellma

# pnpm
pnpm add vellma

# yarn
yarn add vellma
```

### Example: Terminal chat

Let's use the `openai` integration to create a chat model that uses the terminal for user input and output.

```ts
import { openai } from 'vellma/integrations'
import { useChat } from 'vellma/models'
import { terminalIo, useIo } from 'vellma/peripherals'

// Initialize the integration, model, and peripherals.
const integration = openai({ apiKey: 'your-api-key' })
const { factory, model } = useChat({ integration })
const io = useIo(terminalIo())

// Chat loop
while (true) {
  // On each iteration, you will be prompted for input.
  const yourInput = await io.prompt(`You:\n`)
  const yourMessage = factory.human({ text: yourInput })

  // Then, a response will be generated.
  const theirReply = model.generate(yourMessage)

  await io.write(`Assistant:\n`)

  // The response will be written to the terminal in real-time.
  for await (const { textDelta } of theirReply) {
    await io.write(textDelta)
  }

  // Write a final newline to the terminal to prepare for the next iteration.
  await io.write(`\n`)
}
```

Running the code above will allow you to have a basic conversation that looks something like this.

```
You:
Explain human existence in 1 sentence.
Assistant:
Human existence is the intricate dance of biological life, consciousness, relationships, growth, emotions, knowledge, and experience on a small planet in a vast universe.
You:
<Your next prompt here>
```

For more examples, check out the [`playground`](./playground) directory.

## Concepts

To get the best out of `vellma`, there are some concepts that you should understand.

### Interfaces and adapters

In order to keep this library flexible, while also maintaining reasonable defaults, features that relate to _external_ runtime functionality should be implemented with the adapter pattern. In this library, the adapter pattern consists of 2 main concepts: the **interface** and the **adapter**. The interface refers to the _internal_ interface that we will use throughout the codebase. The adapter maps that _internal_ interface to the _external_ interface that a given implementation provides. Some examples of this are:

- Mapping the `openai` endpoint for chat completions to the `ChatIntegration` interface used by the Chat model.
- Mapping the `node:readline` terminal IO utilities to the `IoPeripheral` interface used by features that deal with user input and output.

#### Example: Mapping `api.openai.com/v1/chat/completions` to `ChatIntegration`

Take a look at the following interface for [`ChatIntegration`](./integrations/src/index.ts#7).

```ts
export type ChatIntegration = {
  chat: (messages: ChatMessage[]) => Promise<Consumable<ChatMessage>>,
}
```

The interface is meant to be simple for the generic chat model to consume, so it has a single `chat` property. The `chat` property is a function that takes an array of `ChatMessage` objects (the conversation so far) and returns a single `Consumable<ChatMessage>` object (the reply). The interface for the function that calls `/v1/chat/completions`, however, is a bit more complicated.

```ts
export type OpenAiChatApi = (config: {
  apiKey: string,
  messages: OpenAiChatMessage[],
  model?: string,
  organizationId?: string,
  peripherals?: Partial<Peripherals>,
}) => Promise<OpenAiChatApiResponse>
```

Not only do we need the messages, but we also need the API key, the preferred model, and more. The function is essentially a raw implementation of the corresponding `openai` endpoint, and an adapter must be used to map it to the `ChatIntegration` interface.

### Peripherals

Peripherals wrap environment-specific functionality that we use in our models, integrations, or even other peripherals. Some examples of this are:

- Making HTTP requests
- Getting input from or displaying output to a user
- Storing data temporarily or permanently

#### Example: Getting input from or displaying output to a user

To better understand this concept, take a look at [the `io` implementation under `./peripherals`](./peripherals/src/io/index.ts). The adapter interface is defined by `IoAdapter` as an object that has two async function properties: `read` and `write`. The `terminal` adapter conforms to that interface, and the `useIo` peripheral maps the `terminal` adapter to the `IoPeripheral` interface. This allows us to use the `IoPeripheral` interface throughout the codebase without knowledge about specific implementations that end-users might choose to use. Additionally, we can expose helper functions in the peripheral that utilize the underlying adapter interface without requiring adapters to implement the function directly. The `prompt` function is one example that uses the `write` function to output something to a user followed by the `read` function to receive user input.

### Models

These are the various types of AI models that we can use in our agents. Models are responsible for taking input and producing output. They can be used in a variety of ways, but they are typically used as the higher-level building blocks of an agent. For example, a model might be used to generate a response to a user's input, or it might be used to generate a new piece of content based on a given prompt. Some examples of this are:

- Chat-based LLMs
- Completion-based LLMs
- Text-to-embedding transformers

#### Embedding models

The embedding model is a special type of model that is used to convert text into a vector representation. This is useful for a variety of tasks, including Q&A on a specific dataset. For example, we can generate vector representations of text with the embedding model and then store those vectors in a database alongside the text they represent. Then, we can generate a vector representation of a given input, query the database for the most similar pieces of text, and include one or more of those results in the prompt for the model.

### Integrations

These are the interfaces that allow us to communicate with third-party services. Integrations wrap the functionality of third-party services for use by **models** or **peripherals**. Integrations are organized by provider (e.g. `openai`) and may expose multiple models or peripherals. The interface for an integration is defined by the consumer of the specific implementation. For example, the `openai` integration exposes a `chat` function that conforms to the `ChatIntegration` interface defined by the chat model in [`./models`](./models/src/chat/index.ts).

Example model integrations

- OpenAI
- PaLM 2

Integrations can also be used to wrap peripherals. An integration for `firebase`, for example, could be used by a custom adapter for the `storage` peripheral.

Example storage integrations

- AWS S3
- Google Cloud Storage
- Supabase

## How to contribute to `vellma`

Things are still changing, but I recommend you read through the "Concepts" section above before you get started.

### Set up your development environment

Clone the repo to your machine.

```bash
git clone git@github.com:davidmyersdev/vellma.git
```

Install dependencies with `pnpm`.

```bash
# ~/path/to/vellma
pnpm i
```

Create your `.env` file.

```bash
# ~/path/to/vellma
cp .env.example .env
```

Add your OpenAI API key and (optionally) add your organization and user keys if you have them.

```bash
# ~/path/to/vellma/.env
VITE_OPENAI_API_KEY=your-api-key
# The rest are optional.
VITE_OPENAI_ORGANIZATION_ID=
VITE_OPENAI_USER_ID=
```

Run a playground example `pnpm playground:<example>`. To try out the basic chat implementation, run the following.

```bash
# ~/path/to/vellma
pnpm playground:simple-chat
```

---
_Source: https://npm.io/package/vellma · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._