Skip to content

Model Runtime API

The Model Runtime API manages model catalog records, artifacts, inference engines, and local model services in the Node deployment.

View as Markdown Open the plain-text version of this page.

These endpoints are unavailable in the Cloudflare runtime.

Read Endpoints#

text
GET /v1/projects/:projectId/models/runtime
GET /v1/projects/:projectId/models/activity
GET /v1/projects/:projectId/models/catalog
GET /v1/projects/:projectId/models/:modelId
GET /v1/projects/:projectId/models/:modelId/profiles/:profileId
GET /v1/projects/:projectId/models/:modelId/huggingface-details

Reads require workspace:read.

Catalog Query Parameters#

The catalog endpoint supports the following query parameters:

Parameter Type Notes
search string Free-text search across model names and tags.
category string Filter by category.
pipelineTag string Filter by pipeline tag.
format string Filter by model format.
installed boolean When true, return only installed models.
sort string Sort field and direction.
cursor string Opaque cursor from the previous response.
limit integer Page size.

The response includes the current runtime snapshot plus a catalog page.

Model Detail#

GET /v1/projects/:projectId/models/:modelId returns the model without a specific profile. GET /v1/projects/:projectId/models/:modelId/profiles/:profileId returns the same model detail for the requested profile. Both routes share the same handler.

Catalog And Import#

text
POST /v1/projects/:projectId/models/catalog-refreshes
POST /v1/projects/:projectId/models/imports/huggingface

Import an exact Hugging Face repository:

bash
curl -X POST "$PROVON_API_URL/projects/$PROJECT_ID/models/imports/huggingface" \
  -H "Authorization: Bearer $PROVON_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"repoId": "mlx-community/gemma-4-e2b-it-4bit"}'

Engines#

text
POST   /v1/projects/:projectId/models/engines/:engine/installations
DELETE /v1/projects/:projectId/models/engines/:engine/installations/:generation
DELETE /v1/projects/:projectId/models/engines/:engine

Supported installable engines are llama.cpp, sglang, and vllm.

Engine installation is asynchronous unless the requested engine is already ready.

Artifacts And Services#

text
POST   /v1/projects/:projectId/models/:modelId/profiles/:profileId/downloads
DELETE /v1/projects/:projectId/models/:modelId/profiles/:profileId/downloads/:generation
PUT    /v1/projects/:projectId/models/:modelId/profiles/:profileId/service
DELETE /v1/projects/:projectId/models/:modelId/profiles/:profileId/service
DELETE /v1/projects/:projectId/models/:modelId/profiles/:profileId
POST   /v1/projects/:projectId/models/:modelId/profiles/:profileId/trials

Downloads, engine installations, and service starts expose generation-aware state. Use the runtime and activity endpoints to poll progress instead of assuming that an accepted request is complete.

Trial Endpoint#

The trial endpoint forwards a request to a running model service. It accepts the same OpenAI-compatible paths and bodies as the Gateway, such as /v1/chat/completions or /v1/audio/transcriptions.

bash
curl -X POST "$PROVON_API_URL/projects/$PROJECT_ID/models/$MODEL_ID/profiles/$PROFILE_ID/trials" \
  -H "Authorization: Bearer $PROVON_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "path": "/v1/chat/completions",
    "contentType": "application/json",
    "body": {
      "model": "self/$MODEL_ID",
      "messages": [{"role": "user", "content": "Hello"}]
    }
  }'
Field Type Notes
path string Required. Gateway endpoint path, e.g. /v1/chat/completions.
contentType string Required. application/json or multipart/form-data.
body object Required. Native request body for the selected path.

For multipart/form-data requests, send a standard multipart body and set contentType accordingly.

Mutation endpoints require models:manage or the corresponding project model-management permission. Trial execution requires gateway:invoke or project:data:write.

See AI Models, Self-hosted models, and Models CLI.