Skip to content

Model CLI

Use provon model to search the model catalog, install artifacts, and run a model from the local runtime.

View as Markdown Open the plain-text version of this page.

These commands operate on local runtime state. They do not start the Provon Node server.

Inspect The Runtime#

bash
provon model status
provon model activity

Use PROVON_BLOBS_DIR or XDG_DATA_HOME to control where model weights are stored. Model runtime state (active downloads, running services, and catalog cache) is kept in memory only; it is lost when the CLI process exits.

Find And Import A Model#

Search the installable catalog:

bash
provon model search --query gemma --format gguf

Import an exact Hugging Face repository into the local catalog:

bash
provon model import unsloth/gemma-2-2b-it-GGUF

List installed models or the full catalog:

bash
provon model list
provon model list --all
provon model list --category text --format gguf

Inspect a model:

bash
provon model get <model-id>
provon model get <model-id> --profile <profile-id>

Install A Model#

bash
provon model install <model-id> --profile <profile-id>

The default --engine auto policy checks that the profile's engine family is supported by the remote services/python-inference process and then proceeds to download the model artifacts. It does not install llama.cpp, vLLM, or SGLang on your local machine; those engines must already be available in the Python inference service. Use --engine none to skip the engine check.

Supported engine families are llama.cpp, sglang, and vllm. Whether a profile can be served depends on the engines installed in the remote Python inference service and the model format.

Serve A Model#

bash
provon model serve <model-id> --profile <profile-id> --foreground

Model serving currently stays attached to the CLI process. Stop the process to stop the service. The command reports the self/ Gateway model value when the service is ready.

Remove Artifacts#

bash
provon model remove <model-id> --profile <profile-id>

Environment Variables#

text
PROVON_BLOBS_DIR                  Root data directory; also resolves models/ and data/ defaults
PROVON_MODELS_DIR                 Override the directory for cached model weights
XDG_DATA_HOME                     Fallback data root when PROVON_BLOBS_DIR is unset
PROVON_PYTHON_INFERENCE_URL       Base URL of the python-inference service
PROVON_PYTHON_INFERENCE_API_KEY   API key for the python-inference service
PROVON_TRANSFORMERS_DEVICE        Transformers device override passed to python-inference
PROVON_TRANSFORMERS_DTYPE         Transformers dtype override passed to python-inference
PROVON_TRANSFORMERS_TRUST_REMOTE_CODE=1  Allow remote code in Transformers models