# node-llama-cpp > Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level Latest version **3.22.3** (published 2026-11-05) · MIT license · 0 weekly downloads ## Install ```sh npm install node-llama-cpp pnpm add node-llama-cpp yarn add node-llama-cpp bun add node-llama-cpp ``` Provides the commands `nlc`, `node-llama-cpp`. ## Health **Score 75/100 (B)** — status: active. Positive: has types; esm support; no vulnerabilities; has provenance; recently updated; high maintenance score; high quality score. Warnings: low downloads; large bundle. ## Facts | | | |---|---| | Version | 3.22.3 | | Published | 2026-11-05 | | First published | 2023-08-10 | | Weekly downloads | 0 | | License | MIT | | TypeScript types | bundled | | Module format | ESM + CommonJS | | Node | >=20.0.0 | | Dependencies | 28 | | Unpacked size | 39.4 MB | | Known vulnerabilities | 0 | | Install scripts | yes | | Provenance | attested (GitHub Actions) | | GitHub stars | 2197 | | Author | Gilad S. | | Maintainers | giladgd | | Keywords | llama, llama-cpp, llama.cpp, bindings, ai, cmake, cmake-js, prebuilt-binaries, llm, gguf, metal, cuda, vulkan, grammar, embedding, rerank, reranking, json-grammar, json-schema-grammar, functions, function-calling, structured-decisions, system-one, token-prediction, speculative-decoding, temperature, minP, topK, topP, seed, xtc, json-schema, raspberry-pi, self-hosted, local, catai, mistral, deepseek, qwen, qwq, gpt, gpt-oss, typescript, lora, batching, gpu | ## Links - npm: https://www.npmjs.com/package/node-llama-cpp - Repository: https://github.com/withcatai/node-llama-cpp - Homepage: https://node-llama-cpp.withcat.ai - Issues: https://github.com/withcatai/node-llama-cpp/issues - Funding: https://github.com/sponsors/giladgd - npm.io page: https://npm.io/package/node-llama-cpp ## Dependencies (28) - [ora](https://npm.io/package/ora.md) ^9.3.0 - [bytes](https://npm.io/package/bytes.md) ^3.1.2 - [chalk](https://npm.io/package/chalk.md) ^5.6.2 - [ipull](https://npm.io/package/ipull.md) ^3.9.5 - [which](https://npm.io/package/which.md) ^6.0.1 - [yargs](https://npm.io/package/yargs.md) ^17.7.2 - [ignore](https://npm.io/package/ignore.md) ^7.0.4 - [nanoid](https://npm.io/package/nanoid.md) ^5.1.6 - [semver](https://npm.io/package/semver.md) ^7.7.1 - [chmodrp](https://npm.io/package/chmodrp.md) ^1.0.2 - [env-var](https://npm.io/package/env-var.md) ^7.5.0 - [cmake-js](https://npm.io/package/cmake-js.md) ^8.0.0 - [fs-extra](https://npm.io/package/fs-extra.md) ^11.4.0 - [pretty-ms](https://npm.io/package/pretty-ms.md) ^9.3.0 - [filenamify](https://npm.io/package/filenamify.md) ^6.0.0 - [simple-git](https://npm.io/package/simple-git.md) ^4.0.2 - [slice-ansi](https://npm.io/package/slice-ansi.md) ^8.0.0 - [strip-ansi](https://npm.io/package/strip-ansi.md) ^7.2.0 - [async-retry](https://npm.io/package/async-retry.md) ^1.3.3 - [cross-spawn](https://npm.io/package/cross-spawn.md) ^7.0.6 - [log-symbols](https://npm.io/package/log-symbols.md) ^7.0.1 - [stdout-update](https://npm.io/package/stdout-update.md) ^4.0.1 - [node-addon-api](https://npm.io/package/node-addon-api.md) ^8.9.2 - [lifecycle-utils](https://npm.io/package/lifecycle-utils.md) ^4.5.2 - [proper-lockfile](https://npm.io/package/proper-lockfile.md) ^4.1.2 - [@huggingface/jinja](https://npm.io/package/@huggingface/jinja.md) ^0.5.11 - [is-unicode-supported](https://npm.io/package/is-unicode-supported.md) ^2.1.0 - [validate-npm-package-name](https://npm.io/package/validate-npm-package-name.md) ^7.0.2 ## Alternatives - [@expo/fingerprint](https://npm.io/package/@expo/fingerprint.md) — 6.2M weekly downloads - [@azure/monitor-opentelemetry-exporter](https://npm.io/package/@azure/monitor-opentelemetry-exporter.md) — 850.0K weekly downloads - [@azure/monitor-opentelemetry](https://npm.io/package/@azure/monitor-opentelemetry.md) — 624.0K weekly downloads - [@posthog/ai](https://npm.io/package/@posthog/ai.md) — 423.3K weekly downloads - [fakefilter](https://npm.io/package/fakefilter.md) — 63.9K weekly downloads ## Recent versions - 3.22.3 (latest) — 2026-11-05 - 3.0.0-beta.47 (beta) — 2024-09-23 - 3.22.2 — 2026-10-17 - 3.22.1 — 2026-09-28 - 3.22.0 — 2026-09-27 - 3.21.1 — 2026-09-12 - 3.21.0 — 2026-09-12 - 3.20.0 — 2026-08-11 - 3.19.1 — 2026-07-20 - 3.19.0 — 2026-06-30 - 3.18.1 — 2026-03-17 - 3.18.0 — 2026-03-15 - 3.17.1 — 2026-02-28 - 3.17.0 — 2026-02-27 - 3.16.2 — 2026-02-21 - … 129 more at https://npm.io/package/node-llama-cpp/versions ## README
node-llama-cpp Logo

node-llama-cpp

Run AI models locally on your machine

Pre-built bindings are provided with a fallback to building from source with cmake

[![Build](https://github.com/withcatai/node-llama-cpp/actions/workflows/build.yml/badge.svg)](https://github.com/withcatai/node-llama-cpp/actions/workflows/build.yml) [![License](https://badgen.net/badge/color/MIT/green?label=license)](https://www.npmjs.com/package/node-llama-cpp) [![Types](https://badgen.net/badge/color/TypeScript/blue?label=types)](https://www.npmjs.com/package/node-llama-cpp) [![Version](https://badgen.net/npm/v/node-llama-cpp)](https://www.npmjs.com/package/node-llama-cpp)
✨ [**New:** Structured Decisions with any model](https://node-llama-cpp.withcat.ai/guide/structured-decisions) ✨ ## Features * Run LLMs locally on your machine * [Metal, CUDA and Vulkan support](https://node-llama-cpp.withcat.ai/guide/#gpu-support) * [Pre-built binaries are provided](https://node-llama-cpp.withcat.ai/guide/building-from-source), with a fallback to building from source _**without**_ `node-gyp` or Python * [Adapts to your hardware automatically](https://node-llama-cpp.withcat.ai/guide/#gpu-support), no need to configure anything * A Complete suite of everything you need to use LLMs in your projects * [Use the CLI to chat with a model without writing any code](#try-it-without-installing) * Up-to-date with the latest `llama.cpp`. Download and compile the latest release with a [single CLI command](https://node-llama-cpp.withcat.ai/guide/building-from-source#downloading-a-release) * Enforce a model to generate output in a parseable format, [like JSON](https://node-llama-cpp.withcat.ai/guide/chat-session#json-response), or even force it to [follow a specific JSON schema](https://node-llama-cpp.withcat.ai/guide/chat-session#response-json-schema) * [Provide a model with functions it can call on demand](https://node-llama-cpp.withcat.ai/guide/chat-session#function-calling) to retrieve information or perform actions * [Embedding and reranking support](https://node-llama-cpp.withcat.ai/guide/embedding) * [Structured decisions support](https://node-llama-cpp.withcat.ai/guide/structured-decisions) with any model * [Safe against special token injection attacks](https://node-llama-cpp.withcat.ai/guide/llama-text#input-safety-in-node-llama-cpp) * Great developer experience with full TypeScript support, and [complete documentation](https://node-llama-cpp.withcat.ai/guide/) * Much more ## [Documentation](https://node-llama-cpp.withcat.ai) * [Getting started guide](https://node-llama-cpp.withcat.ai/guide/) * [API reference](https://node-llama-cpp.withcat.ai/api/functions/getLlama) * [CLI help](https://node-llama-cpp.withcat.ai/cli/) * [Blog](https://node-llama-cpp.withcat.ai/blog/) * [Changelog](https://github.com/withcatai/node-llama-cpp/releases) * [Roadmap](https://github.com/orgs/withcatai/projects/1) ## Try It Without Installing Chat with a model in your terminal using [a single command](https://node-llama-cpp.withcat.ai/cli/chat): ```bash npx -y node-llama-cpp chat ``` ## Installation ```bash npm install node-llama-cpp ``` [This package comes with pre-built binaries](https://node-llama-cpp.withcat.ai/guide/building-from-source) for macOS, Linux and Windows. If binaries are not available for your platform, it'll fallback to download a release of `llama.cpp` and build it from source with `cmake`. To disable this behavior, set the environment variable `NODE_LLAMA_CPP_SKIP_DOWNLOAD` to `true`. ## Usage ```typescript import {fileURLToPath} from "url"; import path from "path"; import {getLlama, LlamaChatSession} from "node-llama-cpp"; const __dirname = path.dirname(fileURLToPath(import.meta.url)); const llama = await getLlama(); const model = await llama.loadModel({ modelPath: path.join(__dirname, "models", "Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf") }); const context = await model.createContext(); const session = new LlamaChatSession({ contextSequence: context.getSequence() }); const q1 = "Hi there, how are you?"; console.log("User: " + q1); const a1 = await session.prompt(q1); console.log("AI: " + a1); const q2 = "Summarize what you said"; console.log("User: " + q2); const a2 = await session.prompt(q2); console.log("AI: " + a2); ``` > For more examples, see the [getting started guide](https://node-llama-cpp.withcat.ai/guide/) ## Contributing To contribute to `node-llama-cpp` read the [contribution guide](https://node-llama-cpp.withcat.ai/guide/contributing). ## Acknowledgements * llama.cpp: [ggml-org/llama.cpp](https://github.com/ggml-org/llama.cpp)
Star please

If you like this repo, star it ✨                                                    

--- _Source: https://npm.io/package/node-llama-cpp · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._