0.2.0 • Published 6 months ago
@yankeeinlondon/llm-model v0.2.0
LLM Model
Installation
bun install -g @yankeeinlondon/llm-model
Features
Search
allows easy searching of Hugging Face models from command line
Model Serving
- Run singular models or use speculative sampling with two models running in parallel
allows for interactive input or pure CLI model specification
Benchmarking
- produces tabular format for easy comparison (JSON also available)
- caches results so that quick cross-model views are possible
you can give a partial match for model name or run interactively
List and Lifecycle
- List all local models available for serving
List any currently running servers
stop servers with a
PID
, friendly name reference, or stop all withall
0.2.0
6 months ago