Infer API

One API key. Up to four routes per model.

Infer serves frontier models below each lab’s list price through OpenAI- and Anthropic-compatible endpoints, plus native image and video generation. The suffix on each model ID picks the route; the prepaid wallet and key limits work the same on every route.

Quickstart

Go from a new account to the first API request:

1. Create an account, then confirm the one-time verification link sent by Infer.
2. Open API keys, create an API key, and copy it when shown. Set the copied value as INFER_API_KEY in your local shell or secret manager; never put it in a URL, repository, or chat.
3. Open Wallet and add at least $5 of wallet credit through checkout before sending metered inference.
4. Send the Responses request below. The selected model must show as available on current status.

curl https://infer.flow7.org/v1/responses \
  -H "Authorization: Bearer $INFER_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: first-request" \
  -d '{
    "model": "infer/gpt-5.6-terra:low-cost",
    "input": "Create a typed retry helper with exponential backoff.",
    "max_output_tokens": 1200,
    "relay": {
      "session_id": "repo-acme-42",
      "privacy": "standard"
    }
  }'

Routes

Infer offers a route only when at least one qualified supplier currently passes its model, health, privacy, capability and authorization checks. The live catalog is authoritative for which model IDs are callable now.

Model IDs

A model ID combines the model name and route. Exact model IDs stay within that model family. Dynamic model IDs return the model that actually ran in the response metadata.

API formats

Infer exposes three compatible text-inference request formats over one serving engine, plus native generation endpoints when those models appear in the live catalog. /v1/responses is the canonical Infer/OpenAI Responses surface, /v1/chat/completions accepts OpenAI Chat Completions clients, and /v1/messages accepts Anthropic Messages clients. All three use the same model selectors, wallet, spend limits, routing, privacy checks, prompt-cache accounting, media admission, receipts, and supplier privacy.

All three streaming modes forward upstream text, reasoning/thinking, and tool-call deltas as they arrive. Infer emits keepalives during quiet reasoning periods, does not impose a generation wall-clock timeout while the connection remains alive, and settles final usage, cache accounting, customer charge, and receipt metadata in the terminal stream event. Disconnecting the client cancels the upstream stream and reconciles the wallet reservation rather than leaving a long-running orphan request.

Responses API

Infer returns the normal model response plus the selected route, final charge, token usage, and receipt under relay. The serving supply identity is never returned.

Infer forwards the standard Responses request fields: input, instructions, tools and tool choice, text and structured-output options, reasoning, sampling settings, prompt-cache keys, and metadata. Tools your client runs (function, custom, and shell or patch tools) pass through. Tools a provider would run itself, such as web search, file search, code interpreter, or image generation, are not forwarded, because their charges sit outside token pricing. Infer keeps no server-side conversation state, so send the whole conversation in input: previous_response_id, conversation, stored prompt templates, and background runs return unsupported_parameter. A reasoning.max_tokens budget must be below the output cap; a larger one is dropped and any effort setting is kept.

{
  "id": "resp_…",
  "object": "response",
  "status": "completed",
  "model": "infer/gpt-5.6-terra:low-cost",
  "output_text": "…",
  "usage": {
    "input_tokens": 8240,
    "input_tokens_details": {"cached_tokens": 7100, "cache_write_tokens": 240},
    "output_tokens": 932
  },
  "relay": {
    "receipt_id": "rcpt_…",
    "price_version": "pv_…",
    "tier": "low-cost",
    "resolved_model_class": "GPT-5.6 Terra",
    "cache_status": "partial",
    "environment": "live",
    "provider_disclosed": false,
    "customer_cost_usd": 0.004992
  }
}

Chat Completions compatibility

Existing OpenAI Chat Completions clients can point their base URL at https://infer.flow7.org/v1 and use an Infer model selector. Infer normalizes chat messages, function tools, tool results, structured-output requests, image parts, input audio, and file parts into its canonical request before routing.

curl https://infer.flow7.org/v1/chat/completions \
  -H "Authorization: Bearer $INFER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "infer/gpt-5.6-terra:stable",
    "messages": [{"role": "user", "content": "Return exactly CHAT_OK"}],
    "max_completion_tokens": 32
  }'

stream: true emits chat.completion.chunk SSE and ends with data: [DONE]. stream_options.include_usage is honored. Infer currently supports one choice per request (n=1); audio output, prediction hints, web-search options, and returned logprobs are not part of this compatibility layer.

Anthropic Messages compatibility

The Anthropic SDK can use Infer by setting its base URL to https://infer.flow7.org, passing the Infer API key as x-api-key, and using an Infer model selector. The normal anthropic-version header is accepted. Text, images, documents/PDFs, custom function tools, tool_use/tool_result history, prompt-cache blocks, system prompts, and Anthropic-style streaming events are normalized through the same Infer engine.

curl https://infer.flow7.org/v1/messages \
  -H "x-api-key: $INFER_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "infer/claude-haiku-4-5:official",
    "max_tokens": 32,
    "messages": [{"role": "user", "content": "Return exactly MESSAGES_OK"}]
  }'

stream: true emits Anthropic-compatible message_start, content_block_*, message_delta, and message_stop events. MCP-server and container request features are not currently exposed through Infer's Messages compatibility endpoint.

Native image and video generation

Generation-native models appear only when the live catalog has an authenticated, healthy route and a locked customer price. They currently use the Stable selector because the generation supply is admitted under Infer’s stricter Stable policy. Generation is billed by generated media—not by token rates.

Image generation is synchronous by default and returns either a short-lived Infer-signed URL or b64_json. Send "background": true (URL output only) to generate in the background like a video: the response is 202 with an Infer request ID, and GET /v1/images/{id} reports queued, in_progress, completed with the image link, or failed. A background image is charged once when it completes; a refusal before generation releases the hold at once. Image edits accept JSON public-HTTPS references or one multipart PNG/JPEG/WebP upload up to 20 MiB. stream: true is intentionally rejected until Infer can preserve the upstream terminal event and billing contract without guessing.

curl https://infer.flow7.org/v1/images/generations \
  -H "Authorization: Bearer $INFER_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: image-001" \
  -d '{"model":"infer/gpt-image-2:stable","prompt":"A chrome flower on black glass","quality":"medium","size":"1024x1024"}'

A video request takes a prompt, a duration in seconds, a resolution and, where the model has them, an optional aspect ratio. Each model accepts its own lengths, resolutions and ratios; the live catalog lists them under tiers.stable.generation (durations, the keys of prices, and ratios), and a request outside them is refused before any credit is held. Models with an audio option are generated without sound.

Image-to-video and reference models (their catalog entry has reference_image: "required") need one reference image. Send it as image, a public HTTPS URL, or, where reference_upload is true, upload the file as multipart/form-data with the same fields: one PNG, JPEG or WebP image up to 8 MiB (send a larger one as a URL). Either way the image goes to the model in the same request and Infer does not keep it. A URL is passed on without Infer downloading it, and an upload is sent on and discarded when the request ends. Only a SHA-256 fingerprint is kept, to recognise retries.

curl https://infer.flow7.org/v1/videos \
  -H "Authorization: Bearer $INFER_API_KEY" \
  -H "Idempotency-Key: video-002" \
  -F model=infer/happyhorse-1.1-i2v:stable \
  -F prompt="The steam slowly rises and curls toward the window" \
  -F duration=5 \
  -F resolution=720p \
  -F image=@first-frame.png

Video creation returns 202 with an Infer-owned task ID. Poll /v1/videos/{id}; after completion, download through /v1/videos/{id}/content. Infer also reconciles pending video tasks every five minutes, so terminal settlement does not depend on the client continuing to poll.

curl https://infer.flow7.org/v1/videos \
  -H "Authorization: Bearer $INFER_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: video-001" \
  -d '{"model":"infer/kling-3.0:stable","prompt":"Slow cinematic dolly through mist","duration":5,"resolution":"720p","ratio":"16:9"}'

The full generation ceiling is reserved before dispatch. Infer never automatically retries a paid generation. If a post-dispatch transport outcome is ambiguous, the reservation remains locked rather than assuming the provider did or did not spend. Completed generation receipts expose the customer unit, variant, quantity, and final customer charge while keeping supplier identity and supplier cost private.

How requests are routed

• The API key type, model ID, route and price are locked before the request starts.
• Infer checks model support, context size, privacy needs, current health, media capabilities, and price before choosing a route.
• When a client supplies relay.session_id, the session stays on the same route while that route remains healthy. On supported native Responses routes, Infer derives a private organization-scoped sticky cache key from that session to maximize provider prompt-cache reuse without exposing the raw session ID.
• Retries stay within the selected option’s price and reliability limits.
• The supplier and exact route remain private.

Prompt caching

Prompt cache reads and writes are first-class metered usage. usage.input_tokens_details.cached_tokens reports cache reads and cache_write_tokens reports tokens written to cache. An option without a separate cache-write price bills cache writes at its input rate. Receipts preserve both buckets and the rates applied. For supported Claude Official routes, Infer enables ephemeral prompt caching automatically for standard and no-training requests; explicit cache_control and prompt_cache_key are also accepted when the selected route supports them. Zero-retention requests do not auto-create prompt-cache directives and reject explicit cache controls.

Multimodal input

Infer passes native Responses image, file/PDF, and audio objects through unchanged when the selected route advertises the corresponding modality. Media is never translated into a text-only protocol or silently discarded.

{
  "model": "infer/gpt-5.6-sol:official",
  "input": [{
    "type": "message",
    "role": "user",
    "content": [
      {"type": "input_text", "text": "What is in this image?"},
      {"type": "input_image", "image_url": "https://provider.example/image.png"}
    ]
  }],
  "relay": {"session_id": "vision-session-1", "privacy": "standard"}
}

The live catalog is authoritative: use a selector whose tier runtime reports vision, files, audio, or video for that input type. File/PDF inputs use input_file, audio uses input_audio, and video uses Infer's input_video extension. Chat Completions clients may send video_url content. Infer chooses the verified upstream wire format per capability, so video-capable Official routes may use Chat Completions internally while image/file/audio remain on native Responses. Anthropic Messages does not define a video content block and therefore does not expose video input.

Billing and receipts

Customers fund a prepaid Infer service-credit wallet through secure checkout with no separate Infer checkout fee. The first funding minimum is $5 and later reloads are $50. Applicable tax may be added separately. Wallet credit becomes available only after payment is confirmed. Before a request starts, Infer temporarily holds the maximum estimated cost. After it finishes, Infer charges the actual amount and immediately returns the rest to the wallet. The hold covers the output cap: send max_output_tokens to set it, or Infer sets the output room of the routes serving the model, up to 32,768 tokens. A reply that reaches its cap returns status: "incomplete" and is charged for what was generated, never more than the hold. Where a lab charges more for long prompts, an Official request whose prompt could reach that length is held at the higher rate. If you close a stream early, or a stream is interrupted after output has reached you, the full input and the output generated until then are charged and receipted the same way. Inspect a completed receipt locally.

Errors