# @dithercat/servitor

> simple multiuser LLM chatbot framework

Latest version **0.1.19** (published 2023-06-16) · BSD-2-Clause license · 0 weekly downloads

## Install

```sh
npm install @dithercat/servitor
pnpm add @dithercat/servitor
yarn add @dithercat/servitor
bun add @dithercat/servitor
```

## Health

**Score 25/100 (F)** — status: abandoned.

Positive: has types; esm support; no vulnerabilities.

Warnings: low downloads; pre 1.0.

Negative: abandoned; low maintenance score.

## Facts

| | |
|---|---|
| Version | 0.1.19 |
| Published | 2023-06-16 |
| First published | 2023-06-07 |
| Weekly downloads | 0 |
| License | BSD-2-Clause |
| TypeScript types | bundled |
| Module format | ESM + CommonJS |
| Dependencies | 6 |
| Unpacked size | 661.7 KB |
| Known vulnerabilities | 0 |
| Install scripts | no |
| Author | dither |
| Maintainers | dithercat |

## Links

- npm: https://www.npmjs.com/package/@dithercat/servitor
- Repository: https://github.com/dithercat/servitor
- npm.io page: https://npm.io/package/@dithercat/servitor

## Dependencies (6)

- [openai](https://npm.io/package/openai.md) ^3.2.1
- [sequelize](https://npm.io/package/sequelize.md) ^6.32.0
- [node-fetch](https://npm.io/package/node-fetch.md) ^3.3.1
- [ml-distance](https://npm.io/package/ml-distance.md) ^4.0.1
- [string-format](https://npm.io/package/string-format.md) ^2.0.0
- [cheminfo-types](https://npm.io/package/cheminfo-types.md) ^1.7.2

## Recent versions

- 0.1.19 (latest) — 2023-06-16
- 0.1.18 — 2023-06-12
- 0.1.17 — 2023-06-11
- 0.1.16 — 2023-06-09
- 0.1.15 — 2023-06-09
- 0.1.14 — 2023-06-09
- 0.1.13 — 2023-06-09
- 0.0.12 — 2023-06-08
- 0.0.11 — 2023-06-08
- 0.0.10 — 2023-06-08
- 0.0.9 — 2023-06-07
- 0.0.8 — 2023-06-07
- 0.0.7 — 2023-06-07
- 0.0.6 — 2023-06-07
- 0.0.5 — 2023-06-07
- … 4 more at https://npm.io/package/@dithercat/servitor/versions

## README

<img src="images/logo.png" width="256"/>

# servitor

stupid simple LLM chatbot framework designed for multiuser chats

currently extremely unstable and not really ready for use

if you just want to make a discord LLM bot, see
[ensata](https://github.com/dithercat/ensata), the reference implementation of
servitor

## features

- interchangeable inference/embedding drivers
  - [basilisk](https://github.com/dithercat/basilisk) (recommended)
    - developed in parallel with servitor and supports all the features it uses
  - [text-generation-webui](https://github.com/oobabooga/text-generation-webui)
    and compatible API services (incomplete)
    - does not natively support embeddings, which are required for long-term
      memory
  - OpenAI-compatible API services (incomplete)
    - does not currently support token counting, resulting in inefficient
      context space allocation
  - writing your own driver for something else should be pretty easy
    (see `src/driver/base.ts`)
- dynamic context reallocation
  - context is windowed to 2048 tokens (for LLaMA)
  - if some piece of information (such as long-term memory) is injected into the
    context, then the conversation window shrinks to accomodate it and expands
    again once that information is removed
- universal internal monologue system
  - allows better planning of replies
  - as a side-effect, creates some level of self-consistency for the simulacrum
- vector memory (still ironing this out)
  - persistent driver backed by [pgvector](https://github.com/pgvector/pgvector)
  - simple in-memory driver

## faq

- **who is this for?** - primarily, myself. however, im releasing this publicly
  with the hope that someone else who wants to build a chatbot for i.e. discord
  will find it useful.

- **why not just use langchain?** - it doesnt support multiuser chats, only
  one-on-one conversations between "Human" and "AI". i investigated using it,
  but ultimately i had to roll every part of the stack for this use case myself
  anyway, so it wasnt worth it.

- **why "servitor"?** - this library is named after [the chaos magic concept of
  the same name](https://en.wikipedia.org/wiki/Servitor_(chaos_magic)), because
  the concepts, motivations, and processes involved in the construction of
  servitors operating within the framework of a human psyche has interesting
  parallels with that of agent simulacra operating within the framework of
  humanity's collective psyche as distilled into LLMs.

  and in both cases, as per rule #3 of `sudo`, with great power comes great
  responsibility.

---
_Source: https://npm.io/package/@dithercat/servitor · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._