3.6.0 • Published 5 months ago

node-llama-cpp v3.6.0

Weekly downloads
-
License
MIT
Repository
github
Last release
5 months ago

Build License Types Version

DeepSeek R1 is here!

Features

Documentation

Try It Without Installing

Chat with a model in your terminal using a single command:

npx -y node-llama-cpp chat

Installation

npm install node-llama-cpp

This package comes with pre-built binaries for macOS, Linux and Windows.

If binaries are not available for your platform, it'll fallback to download a release of llama.cpp and build it from source with cmake. To disable this behavior, set the environment variable NODE_LLAMA_CPP_SKIP_DOWNLOAD to true.

Usage

import {fileURLToPath} from "url";
import path from "path";
import {getLlama, LlamaChatSession} from "node-llama-cpp";

const __dirname = path.dirname(fileURLToPath(import.meta.url));

const llama = await getLlama();
const model = await llama.loadModel({
    modelPath: path.join(__dirname, "models", "Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf")
});
const context = await model.createContext();
const session = new LlamaChatSession({
    contextSequence: context.getSequence()
});


const q1 = "Hi there, how are you?";
console.log("User: " + q1);

const a1 = await session.prompt(q1);
console.log("AI: " + a1);


const q2 = "Summarize what you said";
console.log("User: " + q2);

const a2 = await session.prompt(q2);
console.log("AI: " + a2);

For more examples, see the getting started guide

Contributing

To contribute to node-llama-cpp read the contribution guide.

Acknowledgements

3.6.0

5 months ago

3.5.0

6 months ago

3.4.0

7 months ago

3.4.3

6 months ago

3.4.2

6 months ago

3.4.1

6 months ago

3.3.1

8 months ago

3.3.0

8 months ago

3.3.2

7 months ago

3.2.0

9 months ago

3.0.0-beta.45

10 months ago

3.0.0-beta.46

10 months ago

3.0.0-beta.47

10 months ago

3.1.1

10 months ago

3.1.0

10 months ago

3.0.3

10 months ago

3.0.2

10 months ago

3.0.1

10 months ago

3.0.0

10 months ago

2.8.16

11 months ago

3.0.0-beta.23

1 year ago

3.0.0-beta.24

1 year ago

3.0.0-beta.25

1 year ago

3.0.0-beta.26

1 year ago

3.0.0-beta.27

1 year ago

3.0.0-beta.28

1 year ago

3.0.0-beta.29

1 year ago

3.0.0-beta.40

12 months ago

3.0.0-beta.41

12 months ago

3.0.0-beta.42

12 months ago

3.0.0-beta.43

12 months ago

3.0.0-beta.44

12 months ago

2.8.12

1 year ago

2.8.11

1 year ago

2.8.15

12 months ago

2.8.14

12 months ago

2.8.13

12 months ago

3.0.0-beta.30

1 year ago

3.0.0-beta.31

1 year ago

3.0.0-beta.32

1 year ago

3.0.0-beta.33

1 year ago

3.0.0-beta.34

1 year ago

3.0.0-beta.35

1 year ago

3.0.0-beta.36

1 year ago

3.0.0-beta.37

1 year ago

3.0.0-beta.38

1 year ago

3.0.0-beta.39

12 months ago

3.0.0-beta.20

1 year ago

3.0.0-beta.21

1 year ago

3.0.0-beta.22

1 year ago

3.0.0-beta.19

1 year ago

3.0.0-beta.18

1 year ago

2.8.10

1 year ago

3.0.0-beta.17

1 year ago

3.0.0-beta.16

1 year ago

3.0.0-beta.15

1 year ago

2.8.9

1 year ago

3.0.0-beta.14

1 year ago

3.0.0-beta.13

1 year ago

3.0.0-beta.12

1 year ago

2.8.8

1 year ago

3.0.0-beta.11

1 year ago

2.8.7

1 year ago

3.0.0-beta.10

1 year ago

3.0.0-beta.9

1 year ago

2.8.6

1 year ago

3.0.0-beta.7

1 year ago

3.0.0-beta.8

1 year ago

3.0.0-beta.6

1 year ago

3.0.0-beta.5

1 year ago

3.0.0-beta.3

2 years ago

3.0.0-beta.2

2 years ago

3.0.0-beta.4

2 years ago

2.8.5

2 years ago

2.8.4

2 years ago

2.8.3

2 years ago

2.8.2

2 years ago

2.8.1

2 years ago

3.0.0-beta.1

2 years ago

2.5.0

2 years ago

2.7.0

2 years ago

2.6.1

2 years ago

2.6.0

2 years ago

2.5.1

2 years ago

2.7.2

2 years ago

2.6.3

2 years ago

2.8.0

2 years ago

2.7.1

2 years ago

2.6.2

2 years ago

2.7.4

2 years ago

2.7.3

2 years ago

2.7.5

2 years ago

2.4.1

2 years ago

2.4.0

2 years ago

2.3.2

2 years ago

2.3.1

2 years ago

2.3.0

2 years ago

2.2.0

2 years ago

2.1.2

2 years ago

2.1.1

2 years ago

2.1.0

2 years ago

2.0.0

2 years ago

1.3.1

2 years ago

1.3.0

2 years ago

1.2.1

2 years ago

1.2.0

2 years ago

1.1.1

2 years ago

1.1.0

2 years ago

0.0.1

2 years ago

1.0.0

2 years ago