0.1.5 • Published 1 day ago
@huggingface/gguf v0.1.5
@huggingface/gguf
A GGUF parser that works on remotely hosted files.
Spec
Spec: https://github.com/ggerganov/ggml/blob/master/docs/gguf.md
Reference implementation (Python): https://github.com/ggerganov/llama.cpp/blob/master/gguf-py/gguf/gguf_reader.py
Install
npm install @huggingface/gguf
Usage
import { GGMLQuantizationType, gguf } from "@huggingface/gguf";
// remote GGUF file from https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF
const URL_LLAMA = "https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/resolve/191239b/llama-2-7b-chat.Q2_K.gguf";
const { metadata, tensorInfos } = await gguf(URL_LLAMA);
console.log(metadata);
// {
// version: 2,
// tensor_count: 291n,
// kv_count: 19n,
// "general.architecture": "llama",
// "general.file_type": 10,
// "general.name": "LLaMA v2",
// ...
// }
console.log(tensorInfos);
// [
// {
// name: "token_embd.weight",
// shape: [4096n, 32000n],
// dtype: GGMLQuantizationType.Q2_K,
// },
// ... ,
// {
// name: "output_norm.weight",
// shape: [4096n],
// dtype: GGMLQuantizationType.F32,
// }
// ]
Hugging Face Hub
The Hub supports all file formats and has built-in features for GGUF format.
Find more information at: http://hf.co/docs/hub/gguf.
Acknowledgements & Inspirations
- https://github.com/hyparam/hyllama by @platypii (MIT license)
- https://github.com/ahoylabs/gguf.js by @biw @dkogut1996 @spencekim (MIT license)
🔥❤️
0.1.5
1 day ago
0.1.4
2 days ago
0.1.3
5 days ago
0.1.0
9 days ago
0.1.2
9 days ago
0.1.1
9 days ago
0.0.17
22 days ago
0.0.16
27 days ago
0.0.13
28 days ago
0.0.14
28 days ago
0.0.15
27 days ago
0.0.12
28 days ago
0.0.11
1 month ago
0.0.10
1 month ago
0.0.9
2 months ago
0.0.8
2 months ago
0.0.7
2 months ago
0.0.6
2 months ago
0.0.5
2 months ago
0.0.4
2 months ago
0.0.3
2 months ago
0.0.2
2 months ago