Inference server

Libraries and server to build AI applications. Adapters to various native bindings allowing local inference. Integrate it with your application, or use as a microservice.

local ai inference server model pool gpt4all node-llama-cpp transformers.js llama.cpp chatbot bot llm

1.0.0-beta.31 • Published 5 months ago

Libraries and server to build AI applications. Adapters to various native bindings allowing local inference. Integrate it with your application, or use as a microservice.

local ai inference server model pool gpt4all node-llama-cpp transformers.js llama.cpp chatbot bot llm

1.0.0-beta.19 • Published 9 months ago

Inference server Packages

inference-server

lllms