0.2.0 • Published 6 months ago

@yankeeinlondon/llm-model v0.2.0

Weekly downloads
-
License
-
Repository
-
Last release
6 months ago

LLM Model

screenshot

Installation

bun install -g @yankeeinlondon/llm-model

Features

Search

  • allows easy searching of Hugging Face models from command line

    search

Model Serving

  • Run singular models or use speculative sampling with two models running in parallel
  • allows for interactive input or pure CLI model specification

    run

Benchmarking

  • produces tabular format for easy comparison (JSON also available)
  • caches results so that quick cross-model views are possible
  • you can give a partial match for model name or run interactively

    benchmarks

List and Lifecycle

  • List all local models available for serving
  • List any currently running servers

    list

  • stop servers with a PID, friendly name reference, or stop all with all

    stop