Model Fine-Tuning

Node does not spawn Python child processes for training. It calls the Python service over HTTP, which keeps PyTorch, Transformers, and PEFT dependencies out of the Node process.

When To Use It#

Use model fine-tuning when you have curated Dataset examples and want to:

• improve model behavior on a recurring task shape seen in traces;
• produce a project-specific checkpoint that stays inside your infrastructure;
• compare a fine-tuned checkpoint against the base model in the Playground.

Fine-tuning is a follow-up step after Dataset collection. See Datasets for how training examples are produced and validated.

Architecture#

flowchart LR
  dataset["Dataset examples"] --> node["Node runtime"]
  node --> manifest["Dataset manifest"]
  manifest --> python["services/python-fine-tuning"]
  python --> checkpoint["Checkpoint artifact"]
  checkpoint --> runtime["Model runtime"]
  runtime --> gateway["AI Gateway self/ target"]

The Node runtime:

• validates the Dataset schema and example count;
• materializes a training manifest from the Dataset;
• submits the job to services/python-fine-tuning;
• polls for status and stores checkpoint metadata;
• registers completed checkpoints for inference and optionally auto-deploys them.

Supported Training Configuration#

Output format determines the default inference engine when the checkpoint is deployed: Safetensors checkpoints are typically served by vLLM or SGLang; GGUF checkpoints are served by llama.cpp. Compressed quantization produces safetensors checkpoints with compressed-tensor metadata for efficient vLLM/SGLang serving; static FP8 requires a calibration dataset.

• sft uses chat-formatted examples.
• dpo, orpo, and kto use preference-format examples (prompt/chosen/rejected for DPO/ORPO, prompt/completion/label for KTO). These methods are implemented in services/python-fine-tuning but are not yet exposed through the API or CLI; create jobs with method: "sft" for now.

Create A Fine-Tuning Job#

A job references a base model and a Dataset:

curl -X POST "$PROVON_API_URL/projects/$PROJECT_ID/fine-tuning/jobs" \
  -H "Authorization: Bearer $PROVON_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "baseModelId": "<base-model-id>",
    "datasetId": "<dataset-id>",
    "engine": "transformers",
    "method": "sft",
    "adapter": "lora",
    "hyperparameters": {
      "learningRate": 0.0001,
      "numTrainEpochs": 3,
      "perDeviceTrainBatchSize": 1
    },
    "output": {
      "format": "safetensors"
    }
  }'

The base model must already be known to the runtime. The Dataset must use the chat schema with at least one example.

Job Lifecycle#

Jobs move through queued, running, completed, failed, and cancelled. Poll status:

curl "$PROVON_API_URL/projects/$PROJECT_ID/fine-tuning/jobs/$JOB_ID" \
  -H "Authorization: Bearer $PROVON_API_KEY"

Cancel a running job:

curl -X POST "$PROVON_API_URL/projects/$PROJECT_ID/fine-tuning/jobs/$JOB_ID/cancel" \
  -H "Authorization: Bearer $PROVON_API_KEY"

Tuning Configs#

A tuning config is a reusable recipe that binds a base model, a Dataset, hyperparameters, and an output format. You can create configs through the API and reuse them to start jobs, or enable automatic triggers and deployment.

curl -X POST "$PROVON_API_URL/projects/$PROJECT_ID/fine-tuning/tuning-configs" \
  -H "Authorization: Bearer $PROVON_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "baseModelId": "<base-model-id>",
    "datasetId": "<dataset-id>",
    "engine": "transformers",
    "method": "sft",
    "adapter": "lora",
    "hyperparameters": {
      "learningRate": 0.0001,
      "numTrainEpochs": 3
    },
    "output": {
      "format": "safetensors"
    },
    "autoTrigger": {
      "enabled": true,
      "minExampleCount": 10
    },
    "autoDeploy": {
      "enabled": true
    }
  }'

• autoTrigger starts a new job automatically when the Dataset reaches minExampleCount new examples.
• autoDeploy deploys the resulting checkpoint to the model runtime when the job completes successfully.

List, update, and delete configs with the same path pattern. Exact request and response shapes are in the Fine-Tuning API.

Checkpoints#

When a job completes, it produces a checkpoint stored in the project checkpoint registry. List checkpoints:

curl "$PROVON_API_URL/projects/$PROJECT_ID/fine-tuning/checkpoints" \
  -H "Authorization: Bearer $PROVON_API_KEY"

Deploy a checkpoint to the model runtime:

curl -X POST "$PROVON_API_URL/projects/$PROJECT_ID/fine-tuning/checkpoints/$CHECKPOINT_ID/deploy" \
  -H "Authorization: Bearer $PROVON_API_KEY"

Deployment loads the checkpoint into the model runtime and makes it available as a self/ Gateway target. The checkpoint must have a registered inference profile.

Lineage#

Trace the provenance of a model:

curl "$PROVON_API_URL/projects/$PROJECT_ID/fine-tuning/lineage/$ROOT_MODEL_ID" \
  -H "Authorization: Bearer $PROVON_API_KEY"

Lineage records the chain of base models, fine-tuning jobs, and checkpoints.

Export Checkpoints#

Export a checkpoint or a Hugging Face model to the Hugging Face Hub:

curl -X POST "$PROVON_API_URL/projects/$PROJECT_ID/fine-tuning/export" \
  -H "Authorization: Bearer $PROVON_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "sourceType": "checkpoint",
    "checkpointId": "<checkpoint-id>",
    "format": "gguf",
    "quantMethod": "q4_k_m",
    "destination": "huggingface-hub",
    "hfRepoId": "owner/repo"
  }'

Supported export formats are gguf and merged-16bit. lora-adapter export is not currently supported by the API.

Export requires a configured Hugging Face connector with a valid access token. The token is resolved from the project connector and passed to the export worker securely; it is not persisted on the job record.

Configuration#

Node runtime variables that control the fine-tuning and export background loops:

The Python fine-tuning service is configured separately from Node:

Node and the Python service must see the training work directory at the same absolute path, because training requests pass filesystem paths for the Dataset manifest and checkpoint output. When running on different hosts, mount or replicate the configured PROVON_MODEL_FINE_TUNING_WORK_DIR so both processes resolve the same paths.

Capability Requirements#

The project API key needs:

• workspace:read or project:data:read to list jobs, checkpoints, and lineage;
• models:manage or project:data:write to create jobs, cancel jobs, and export checkpoints;
• models:manage or project:models:manage to deploy checkpoints.

Limitations#

• Fine-tuning is only available in the Node runtime. Cloudflare Workers do not provide local training or checkpoint storage.
• The current supported engine set is transformers; others may be added over time.
• GGUF export requires a local checkout of llama.cpp with convert_hf_to_gguf.py and the llama-quantize binary.
• Compressed quantization (FP8/INT8) requires the compressed optional dependency (llm-compressor) in the Python fine-tuning service.