# Model Fine-Tuning Provon can train adapters and small models from project Datasets and serve the resulting checkpoints behind the AI Gateway. The Node runtime owns job scheduling, checkpoint tracking, and deployment; the actual training runs in `services/python-fine-tuning`, a standalone Python service that Node controls remotely over HTTP. Node does **not** spawn Python child processes for training. It calls the Python service over HTTP, which keeps PyTorch, Transformers, and PEFT dependencies out of the Node process. ## When To Use It Use model fine-tuning when you have curated Dataset examples and want to: - improve model behavior on a recurring task shape seen in traces; - produce a project-specific checkpoint that stays inside your infrastructure; - compare a fine-tuned checkpoint against the base model in the Playground. Fine-tuning is a follow-up step after Dataset collection. See [Datasets](../datasets/index.md) for how training examples are produced and validated. ## Architecture ```mermaid flowchart LR dataset["Dataset examples"] --> node["Node runtime"] node --> manifest["Dataset manifest"] manifest --> python["services/python-fine-tuning"] python --> checkpoint["Checkpoint artifact"] checkpoint --> runtime["Model runtime"] runtime --> gateway["AI Gateway self/ target"] ``` The Node runtime: - validates the Dataset schema and example count; - materializes a training manifest from the Dataset; - submits the job to `services/python-fine-tuning`; - polls for status and stores checkpoint metadata; - registers completed checkpoints for inference and optionally auto-deploys them. ## Supported Training Configuration | Dimension | Supported values | | ----------------------- | ---------------------------------------------------------------------------------- | | Engine | `transformers` | | Method | `sft` (API/CLI current); `dpo`, `orpo`, `kto` implemented in Python service only | | Adapter | `lora`, `qlora`, `full` | | Output format | `safetensors`, `gguf` | | GGUF quantization | `q4_0`, `q4_k_m`, `q4_k_s`, `q5_0`, `q5_k_m`, `q6_k`, `q8_0`, `f16`, `bf16`, `f32` | | Compressed quantization | `fp8`, `fp8_dynamic`, `fp8_static`, `int8` | Output format determines the default inference engine when the checkpoint is deployed: Safetensors checkpoints are typically served by vLLM or SGLang; GGUF checkpoints are served by llama.cpp. Compressed quantization produces safetensors checkpoints with compressed-tensor metadata for efficient vLLM/SGLang serving; static FP8 requires a calibration dataset. - `sft` uses chat-formatted examples. - `dpo`, `orpo`, and `kto` use preference-format examples (`prompt`/`chosen`/`rejected` for DPO/ORPO, `prompt`/`completion`/`label` for KTO). These methods are implemented in `services/python-fine-tuning` but are not yet exposed through the API or CLI; create jobs with `method: "sft"` for now. ## Create A Fine-Tuning Job A job references a base model and a Dataset: ```bash curl -X POST "$PROVON_API_URL/projects/$PROJECT_ID/fine-tuning/jobs" \ -H "Authorization: Bearer $PROVON_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "baseModelId": "", "datasetId": "", "engine": "transformers", "method": "sft", "adapter": "lora", "hyperparameters": { "learningRate": 0.0001, "numTrainEpochs": 3, "perDeviceTrainBatchSize": 1 }, "output": { "format": "safetensors" } }' ``` The base model must already be known to the runtime. The Dataset must use the `chat` schema with at least one example. ## Job Lifecycle Jobs move through `queued`, `running`, `completed`, `failed`, and `cancelled`. Poll status: ```bash curl "$PROVON_API_URL/projects/$PROJECT_ID/fine-tuning/jobs/$JOB_ID" \ -H "Authorization: Bearer $PROVON_API_KEY" ``` Cancel a running job: ```bash curl -X POST "$PROVON_API_URL/projects/$PROJECT_ID/fine-tuning/jobs/$JOB_ID/cancel" \ -H "Authorization: Bearer $PROVON_API_KEY" ``` ## Tuning Configs A tuning config is a reusable recipe that binds a base model, a Dataset, hyperparameters, and an output format. You can create configs through the API and reuse them to start jobs, or enable automatic triggers and deployment. ```bash curl -X POST "$PROVON_API_URL/projects/$PROJECT_ID/fine-tuning/tuning-configs" \ -H "Authorization: Bearer $PROVON_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "baseModelId": "", "datasetId": "", "engine": "transformers", "method": "sft", "adapter": "lora", "hyperparameters": { "learningRate": 0.0001, "numTrainEpochs": 3 }, "output": { "format": "safetensors" }, "autoTrigger": { "enabled": true, "minExampleCount": 10 }, "autoDeploy": { "enabled": true } }' ``` - `autoTrigger` starts a new job automatically when the Dataset reaches `minExampleCount` new examples. - `autoDeploy` deploys the resulting checkpoint to the model runtime when the job completes successfully. List, update, and delete configs with the same path pattern. Exact request and response shapes are in the [Fine-Tuning API](../api/fine-tuning.md). ## Checkpoints When a job completes, it produces a checkpoint stored in the project checkpoint registry. List checkpoints: ```bash curl "$PROVON_API_URL/projects/$PROJECT_ID/fine-tuning/checkpoints" \ -H "Authorization: Bearer $PROVON_API_KEY" ``` Deploy a checkpoint to the model runtime: ```bash curl -X POST "$PROVON_API_URL/projects/$PROJECT_ID/fine-tuning/checkpoints/$CHECKPOINT_ID/deploy" \ -H "Authorization: Bearer $PROVON_API_KEY" ``` Deployment loads the checkpoint into the model runtime and makes it available as a `self/` Gateway target. The checkpoint must have a registered inference profile. ## Lineage Trace the provenance of a model: ```bash curl "$PROVON_API_URL/projects/$PROJECT_ID/fine-tuning/lineage/$ROOT_MODEL_ID" \ -H "Authorization: Bearer $PROVON_API_KEY" ``` Lineage records the chain of base models, fine-tuning jobs, and checkpoints. ## Export Checkpoints Export a checkpoint or a Hugging Face model to the Hugging Face Hub: ```bash curl -X POST "$PROVON_API_URL/projects/$PROJECT_ID/fine-tuning/export" \ -H "Authorization: Bearer $PROVON_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "sourceType": "checkpoint", "checkpointId": "", "format": "gguf", "quantMethod": "q4_k_m", "destination": "huggingface-hub", "hfRepoId": "owner/repo" }' ``` Supported export formats are `gguf` and `merged-16bit`. `lora-adapter` export is not currently supported by the API. Export requires a configured Hugging Face connector with a valid access token. The token is resolved from the project connector and passed to the export worker securely; it is not persisted on the job record. ## Configuration Node runtime variables that control the fine-tuning and export background loops: | Variable | Default | Purpose | | -------------------------------------- | ----------------------- | ----------------------------------------------- | | `PROVON_PYTHON_FINE_TUNING_URL` | `http://127.0.0.1:8001` | Base URL of `services/python-fine-tuning` | | `PROVON_PYTHON_FINE_TUNING_API_KEY` | none | Bearer token shared with Node | | `PROVON_MODEL_FINE_TUNING_WORK_DIR` | none | Root for training temporary files and manifests | | `PROVON_MODEL_EXPORT_WORK_DIR` | none | Root for export temporary files | | `PROVON_MODEL_FINE_TUNING_INTERVAL_MS` | 5000 | Poll interval for the fine-tuning consumer | | `PROVON_MODEL_FINE_TUNING_LOCK_MS` | 1,800,000 (30 min) | Lease duration for a fine-tuning job | | `PROVON_MODEL_EXPORT_INTERVAL_MS` | 5000 | Poll interval for the export consumer | | `PROVON_MODEL_EXPORT_LOCK_MS` | 3,600,000 (1 hour) | Lease duration for an export job | The Python fine-tuning service is configured separately from Node: | Variable | Default | Purpose | | ----------------------- | ----------- | ------------------------------------------ | | `PROVON_HOST` | `0.0.0.0` | Bind host | | `PROVON_PORT` | `8001` | Bind port | | `PROVON_API_KEY` | none | Bearer token shared with Node | | `PROVON_LOG_LEVEL` | `info` | Service log level | | `PROVON_LLAMA_CPP_PATH` | `llama.cpp` | Path to llama.cpp checkout for GGUF export | Node and the Python service must see the training work directory at the same absolute path, because training requests pass filesystem paths for the Dataset manifest and checkpoint output. When running on different hosts, mount or replicate the configured `PROVON_MODEL_FINE_TUNING_WORK_DIR` so both processes resolve the same paths. ## Capability Requirements The project API key needs: - `workspace:read` or `project:data:read` to list jobs, checkpoints, and lineage; - `models:manage` or `project:data:write` to create jobs, cancel jobs, and export checkpoints; - `models:manage` or `project:models:manage` to deploy checkpoints. ## Limitations - Fine-tuning is only available in the Node runtime. Cloudflare Workers do not provide local training or checkpoint storage. - The current supported engine set is `transformers`; others may be added over time. - GGUF export requires a local checkout of llama.cpp with `convert_hf_to_gguf.py` and the `llama-quantize` binary. - Compressed quantization (FP8/INT8) requires the `compressed` optional dependency (`llm-compressor`) in the Python fine-tuning service.