# hyllama

> llama.cpp gguf file parser for javascript

Latest version **0.2.2** (published 2024-04-19) · MIT license · 0 weekly downloads

## Install

```sh
npm install hyllama
pnpm add hyllama
yarn add hyllama
bun add hyllama
```

## Health

**Score 30/100 (F)** — status: abandoned.

Positive: has types; esm support; no vulnerabilities; high quality score.

Warnings: low downloads; pre 1.0.

Negative: abandoned; low maintenance score.

## Facts

| | |
|---|---|
| Version | 0.2.2 |
| Published | 2024-04-19 |
| First published | 2023-12-27 |
| Weekly downloads | 0 |
| License | MIT |
| TypeScript types | bundled |
| Module format | ESM + CommonJS |
| Dependencies | 0 |
| Unpacked size | 9.6 KB |
| Known vulnerabilities | 0 |
| Install scripts | no |
| GitHub stars | 51 |
| Maintainers | platypii |
| Keywords | llama, llama.cpp, llama-cpp, llamacpp, llm, gguf, parser, machine learning, ml |

## Links

- npm: https://www.npmjs.com/package/hyllama
- Repository: https://github.com/hyparam/hyllama
- Homepage: https://github.com/hyparam/hyllama#readme
- Issues: https://github.com/hyparam/hyllama/issues
- npm.io page: https://npm.io/package/hyllama

## Alternatives

- [monaco-yaml](https://npm.io/package/monaco-yaml.md) — 420.1K weekly downloads
- [@crewx/workflow](https://npm.io/package/@crewx/workflow.md) — 3.1K weekly downloads
- [yaml-cat](https://npm.io/package/yaml-cat.md) — 38 weekly downloads
- [nunjucks-in-yaml](https://npm.io/package/nunjucks-in-yaml.md) — 9 weekly downloads
- [shopify-symlinks](https://npm.io/package/shopify-symlinks.md) — 3 weekly downloads

## Recent versions

- 0.2.2 (latest) — 2024-04-19
- 0.2.1 — 2024-03-27
- 0.2.0 — 2024-03-09
- 0.1.2 — 2024-01-03
- 0.1.1 — 2023-12-28
- 0.1.0 — 2023-12-27

## README

# hyllama

![hyllama](hyllama.jpg)

[![npm](https://img.shields.io/npm/v/hyllama)](https://www.npmjs.com/package/hyllama)
[![workflow status](https://github.com/hyparam/hyllama/actions/workflows/ci.yml/badge.svg)](https://github.com/hyparam/hyllama/actions)
[![mit license](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
![dependencies](https://img.shields.io/badge/Dependencies-0-blueviolet)

Javascript parser for [llama.cpp](https://github.com/ggerganov/llama.cpp) gguf files.

This library makes it easy to parse metadata from GGUF files.

llama.cpp was originally an implementation of meta's llama model in C++, particularly on apple m-series chips.
But it has quickly evolved into a powerful tool for running various trained LLM models, on cpu or gpu.
The runtime has minimal dependencies and so is easy to deploy.
Model files are frequently distributed as .gguf files which contain all the info needed to run a model including architecture and weights.
[TheBloke](https://huggingface.co/TheBloke) provides a great collection of serialized gguf model files, at varying levels of quantization.

Model files are often very large.
A goal of this library is to parse the file efficiently, without loading the entire file into memory.

Dependency free since 2023!

## Installation

```bash
npm install hyllama
```

## Usage

### Node.js

If you're in a node.js environment, you can load a .gguf file with the following example:

```js
const { ggufMetadata } = await import('hyllama')
const fs = await import('fs')

// Read first 10mb of gguf file
const fd = fs.openSync('example.gguf', 'r')
const buffer = new Uint8Array(10_000_000)
fs.readSync(fd, buffer, 0, 10_000_000, 0)
fs.closeSync(fd)

// Parse metadata and tensor info
const { metadata, tensorInfos } = ggufMetadata(buffer.buffer)
```

### Browser

If you're in a browser environment, you'll probably get .gguf file data from either a drag-and-dropped file from the user, or downloaded from the web.

To load .gguf data in the browser from a remote `url`, it is recommended that you use an [HTTP range request](https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests) to get just the first bytes:

```js
import { ggufMetadata } from 'hyllama'

const headers = new Headers({ Range: 'bytes=0-10000000' })
const res = await fetch(url, { headers })
const arrayBuffer = await res.arrayBuffer()
const { metadata, tensorInfos } = ggufMetadata(arrayBuffer)
```

To parse .gguf files from a user drag-and-drop action, see example in [index.html](index.html).

## File Size

Since .gguf files are typically very large, it is recommended that you only load the start of the file that contains the metadata.
How many bytes you need for the metadata depends on the gguf file.
In practice, most .gguf files have metadata that takes up a few megabytes.
If you get an error "RangeError: Offset is outside the bounds of the DataView" then you probably didn't fetch enough bytes.

## References

 - https://github.com/ggerganov/llama.cpp
 - https://github.com/ggerganov/ggml/blob/master/docs/gguf.md
 - https://huggingface.co/TheBloke

---
_Source: https://npm.io/package/hyllama · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._