Skip to content

AI Models

Provon can import, run, fine-tune, and deploy open-weight models behind the same AI Gateway contract used for cloud providers.

View as Markdown Open the plain-text version of this page.

AI Models are available in the Node deployment. The Cloudflare runtime can route to remote providers but does not run local inference or training workloads.

Choose A Model Path#

Goal Start with
Run a local model for the first time Quickstart
Use a managed or BYOK cloud model Model providers
Import and serve a local or private model Self-hosted models
Train a checkpoint from a reviewed Dataset Model fine-tuning
Reuse a training recipe with auto-trigger/deploy Tuning configs
Export a checkpoint to Hugging Face Hub Export checkpoints
Install and run a model directly from the command line Models CLI
Diagnose local model problems Troubleshooting
Route traffic to a running model Gateway routing

Runtime Lifecycle#

flowchart LR
  catalog["Model catalog"] --> import["Import model"]
  import --> install["Install profile"]
  install --> serve["Start inference"]
  serve --> gateway["Gateway self/ target"]
  dataset["Reviewed Dataset"] --> config["Tuning config"]
  config --> tune["Fine-tuning job"]
  tune --> checkpoint["Checkpoint"]
  checkpoint --> export["Export to Hub"]
  checkpoint --> serve

The Node runtime owns model records, artifacts, download lifecycle, and service coordination. Inference runs in services/python-inference; training runs in services/python-fine-tuning. Engines are provided by the Python inference service, not installed by Node.

Base Models And Checkpoints#

A base model is imported from a catalog source such as Hugging Face Hub. A runtime profile describes the artifact format and compatible inference engine.

A fine-tuning job references a base model, an immutable Dataset manifest, and an optional tuning config that captures reusable hyperparameters and auto-deploy policy. Successful jobs produce checkpoints with lineage back to both inputs. Deploying a checkpoint registers an inference profile and exposes it as a Gateway target. Checkpoints can also be exported to Hugging Face Hub in GGUF or merged-16bit format.

Deployment Boundary#

Running an AI model is different from self-hosting the Provon platform:

  • Self-hosting Provon determines where the Workbench, APIs, telemetry, and jobs run.
  • Self-hosting a model provides one private upstream target for the AI Gateway.

The two can run on different hosts when network and credential boundaries permit it.

  1. Confirm host and engine support.
  2. Import one exact model and install one compatible profile.
  3. Serve it locally and send a direct provider-qualified request.
  4. Verify the resulting trace before adding routing policy.
  5. Fine-tune only from a reviewed Dataset.
  6. Deploy a checkpoint behind a stable logical model name.
  7. Compare new traces with the behavior that motivated the change.