Gateway Limits And Guardrails

• Usage limits decide whether the project API key may consume more requests, tokens, or cost.
• Guardrails inspect the request or response and monitor, adjust, or block supported conditions.

Both controls produce structured project evidence: usage-policy events or guardrail trace events. Neither replaces application authorization or provider-side safety controls.

Usage limits#

Usage limits bound how much a Gateway workload can consume. A limit is scoped to an organization, project, or API key, and can optionally be narrowed to a specific request model id. The runtime checks all matching limits before routing and rejects the request when any limit would be exceeded.

Limit scopes#

The narrowest matching scope wins first. For example, a model_per_api_key limit on one model id is checked independently from a project-level token budget.

Metrics and modes#

Supported windows are minute, hour, day, week, month, quarter, year, custom, and lifetime.

Configure a limit#

In the Workbench:

1. Open API keys.
2. Expand the project key used by the workload.
3. Under Limits, select Add limit.
4. Choose metric, type, limit, period, and scope.
5. For a model-specific scope, enter the exact request model id.
6. Save the limit and monitor used, reserved, remaining, and reset values.

The runtime reserves estimated usage before routing, then settles successful requests against actual usage when the provider reports it. Failed requests release their reservation.

Rejection contract#

When admission would exceed a policy, the Gateway returns 429:

{
  "error": {
    "code": "gateway_budget_exceeded",
    "message": "Gateway budget exceeded",
    "reason": "cost_budget_exceeded",
    "policyId": "policy_123",
    "resetAt": "2026-08-02T00:00:00.000Z"
  }
}

Rate-limit rejections use rate_limit_exceeded. Responses include:

Retry-After
RateLimit-Limit
RateLimit-Remaining
RateLimit-Reset

Do not implement automatic retries for a cost or lifetime budget rejection. Retry only when the policy has a reset time and the workload can wait.

Success checks#

• The saved rule appears under Limits with the expected scope, metric, and window.
• A request at the limit returns 429 with the matching code, and the trace records the policy id that rejected it.
• Removing or widening the rule allows the request again without restarting the Gateway.

Guardrail modes and actions#

Guardrails are configured under Guardrails and run in configured priority order.

Modes#

Actions#

When a rule matches in Enforce mode, the Gateway can take one of the following actions:

On an input-direction match, abort_stream is treated as block. Start new rules with Monitor against representative traffic. Review the captured match evidence before switching to Degrade, Block, or Abort stream.

In Monitor mode the trace still records the action the rule would have taken (as provon.guard.would_action), so you can audit false positives before enforcement.

Built-in guardrails#

Data protection#

These rules support monitoring, redaction through Degrade, and blocking. Match evidence stores categories and hashes rather than the matched secret value.

Semantic policy#

The semantic rule calls a checker model: you write one context policy, and the checker judges the selected directions against it. It is a single policy surface, not a set of prebuilt scenario checks.

Configuration:

• Directions — check the request input, the current output, or both.
• Checker model — any Gateway model route your project can reach. The checker runs as an ordinary Gateway model call; its own calls are not re-evaluated by guardrails.
• Policy — the admin-written context policy the checker judges against.
• Policy ID — optional category stored as details.category on matches.
• Bounded check payload: segment buffer 262144 bytes, segment window 30000 ms, checker timeout 15000 ms, context budget 12000 characters, checker output 512 tokens. When the checker model declares a context window, the actual context budget is the smaller of the admin cap and the window-derived budget.

Only Monitor and Block are valid actions for this rule; the management API rejects degrade or abort_stream with 400. Start in Monitor against representative traffic.

Scope and recording:

• The checker sees only what the Gateway owns: the current output's text and tool-call arguments, plus assembled conversation context. System and developer instructions and every user message are kept whole; the context budget trims only older assistant actions, tool results, and tool names, oldest first. Trimmed amounts are recorded as counts on the match evidence — never as content. The semantic rule does not control the host's MCP servers, browser, shell, or filesystem.
• A hit or an unavailable checker records a guardrail decision in the same pipeline as every other rule; one request can produce one decision event per evaluated rule, and the first blocking decision decides whether the exchange is intercepted.
• In Monitor mode the exchange passes through and the decision merges into the request span. In Enforce mode an output check buffers each segment until the checker returns; complete outputs are never truncated — the caps bound what is sent to the checker, not what reaches the caller.
• Match evidence carries the checker reason, buffered bytes, buffering duration, and trimmed counts; it never contains the request, output, or policy text.
• In Enforce mode a checker failure, timeout, or oversized input records the decision and rejects the exchange; in Monitor mode the same conditions only record it. A rejected exchange does not ban the session or API key; the next request is evaluated fresh.

Supported endpoint shapes are chat completions, responses, and messages. On unsupported endpoint shapes, Enforce returns 409 rather than passing the exchange unchecked; upstream non-2xx responses pass through unchanged with fixed guardrail evidence.

Tool governance#

Tool argument policies support wildcard tool names and optional argument-path predicates. Use them to enforce explicit application policy, not to infer business authorization from prompt text.

Context budget#

Input token checks are estimates. API-key token policies perform their own admission reservation and settle against provider-reported usage when available.

Guardrail block contract#

An enforced block returns 409:

{
  "error": {
    "message": "Provon Gateway blocked this exchange because a guardrail was triggered.",
    "type": "provon_gateway_guard",
    "code": "PROVON_GATEWAY_GUARD_TRIGGERED",
    "param": null,
    "retryable": false,
    "details": {
      "pluginId": "provon.guard.secret-leak",
      "action": "block",
      "direction": "input",
      "category": "openai_api_key"
    }
  }
}

The response includes:

x-provon-guard-triggered: true
x-provon-guard-plugin-id
x-provon-guard-action
x-provon-request-id

For semantic matches, details.category carries the configured Policy ID and the checker reason lives in the captured trace evidence; the error details object does not grow extra fields.

Do not retry the same blocked payload. Change the request or the project policy.

Tool argument policies#

Each tool argument policy rule matches:

• a tool-name pattern;
• an optional argument field path;
• an operator;
• an optional comparison value;
• an optional reason and category.

Supported operators include existence, equality, numeric comparison, string containment, prefix/suffix matching, and allow/deny list membership.

Example intent:

tool pattern: shell_*
field:        command
operator:     contains
value:        rm -rf
action:       Block

Keep tool authorization in the application and the MCP policy layer. A Gateway guardrail is an additional policy boundary over model-call content, not the source of user permissions.

Layering controls#

Apply controls in this order:

1. Use application authentication and authorization for user actions.
2. Restrict project API-key capabilities and rotate keys independently.
3. Add API-key usage limits for hard resource boundaries.
4. Add guardrails for request, response, and tool-shape policy.
5. Restrict provider keys and model mappings to approved upstream targets.
6. Use Trace Capture to retain the evidence needed to audit decisions.

A request may be admitted by usage policy and still be blocked by a guardrail. A request may pass both and still fail provider authentication, capability checks, or upstream execution.

Security guidance#

• Give workloads Provon project keys, never upstream provider keys.
• Use a separate project key per environment or application boundary.
• Avoid raw secrets and unnecessary personal data in prompts and context headers.
• Use Specific model mappings when a provider key must expose a strict allowlist.
• Treat header overrides and custom upstream URLs as privileged configuration.
• Review provider key, guardrail, and API-key changes as production policy changes.
• Apply project telemetry retention to captured prompts, responses, and tool arguments.