llama-cpp-pro

llama-cpp-pro

A native Capacitor plugin that embeds llama.cpp directly into mobile apps, enabling offline AI inference with comprehensive support for text generation, multimodal processing, TTS, LoRA adapters, and more.

Annadata.ai: Inference of LLaMA model in pure C/C++ used in Annadata.ai

Features

• Offline AI Inference: Run large language models completely offline on mobile devices
• Text Generation: Complete text completion with streaming support
• Chat Conversations: Multi-turn conversations with context management
• Multimodal Support: Process images and audio alongside text
• Text-to-Speech (TTS): Generate speech from text using vocoder models
• LoRA Adapters: Fine-tune models with LoRA adapters
• Embeddings: Generate vector embeddings for semantic search
• Reranking: Rank documents by relevance to queries
• Session Management: Save and load conversation states
• Benchmarking: Performance testing and optimization tools
• Structured Output: Generate JSON with schema validation
• Cross-Platform: iOS, Android, Web/PWA, and Desktop (Windows, macOS, Linux) with native optimizations

Platform Support

Feature	iOS	Android	Web (PWA)	Desktop
Text Generation
Chat Conversations
Streaming				¹
Multimodal			²
TTS			²
LoRA Adapters			²
Embeddings
Reranking			³
Session Management			⁴
Benchmarking
GPU Acceleration	Metal	CPU/Adreno	—	Vulkan/CUDA/ROCm/Metal

¹ Desktop: SSE streaming from the native sidecar (/v1/chat/completions, /v1/completions with stream: true).
² Web: auxiliary GGUF files must be staged in WASM VFS.
³ Web: requires rank-pooling embedding model.
⁴ Web: sessions persist in worker MEMFS for tab lifetime.

---

Builds

# iOS + Android + PWA (npm release / Capacitor)
./build-variants.sh --variant minimal

# iOS only / Android only
./build-variants.sh --variant ios-only
./build-variants.sh --variant android-only

# Desktop / Electron (macOS universal sidecar: arm64 + x64)
./build-variants.sh --variant desktop
./build-variants.sh --variant minimal --with-desktop --desktop-arch=universal

See BUILD_GUIDE.md and README_BUILD_SYSTEM.md for full build, API, and troubleshooting details.

Contributing

We welcome contributions! Please see our Contributing Guide for details.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

• llama.cpp - The core inference engine
• Capacitor - The cross-platform runtime
• Annadata.ai - Complete system developed and powered by npm

Support

• Email: support@arusatech.com ; yakub@annadata.ai
• Issues: GitHub Issues
• Documentation: GitHub Wiki