@dithercat/servitor NPM

servitor

stupid simple LLM chatbot framework designed for multiuser chats

currently extremely unstable and not really ready for use

if you just want to make a discord LLM bot, see ensata, the reference implementation of servitor

interchangeable inference/embedding drivers
- basilisk (recommended)
  - developed in parallel with servitor and supports all the features it uses
- text-generation-webui and compatible API services (incomplete)
  - does not natively support embeddings, which are required for long-term memory
- OpenAI-compatible API services (incomplete)
  - does not currently support token counting, resulting in inefficient context space allocation
- writing your own driver for something else should be pretty easy (see src/driver/base.ts)
dynamic context reallocation
- context is windowed to 2048 tokens (for LLaMA)
- if some piece of information (such as long-term memory) is injected into the context, then the conversation window shrinks to accomodate it and expands again once that information is removed
universal internal monologue system
- allows better planning of replies
- as a side-effect, creates some level of self-consistency for the simulacrum
vector memory (still ironing this out)
- persistent driver backed by pgvector
- simple in-memory driver

who is this for? - primarily, myself. however, im releasing this publicly with the hope that someone else who wants to build a chatbot for i.e. discord will find it useful.
why not just use langchain? - it doesnt support multiuser chats, only one-on-one conversations between "Human" and "AI". i investigated using it, but ultimately i had to roll every part of the stack for this use case myself anyway, so it wasnt worth it.
why "servitor"? - this library is named after the chaos magic concept of the same name, because the concepts, motivations, and processes involved in the construction of servitors operating within the framework of a human psyche has interesting parallels with that of agent simulacra operating within the framework of humanity's collective psyche as distilled into LLMs.
and in both cases, as per rule #3 of sudo, with great power comes great responsibility.