by Flow7
Sign inCreate account

Infer × LiteLLM · setup guide for 1.106.4

Call Infer through LiteLLM.

Use the openai/ prefix with api_base=https://infer.flow7.org/v1, in the SDK or the proxy. LiteLLM strips the prefix once, so Infer receives the selector unchanged.

Set up in one commandGenerate the config
libraryLiteLLM 1.106.4
routeopenai/ · Chat Completions
base URLhttps://infer.flow7.org/v1
keyINFER_API_KEY
selectorloading catalog…

Infer Setup 0.2.0

Two commands.

Pick a model from the live list; the export prints the proxy config for it. Nothing is sent to a model.

npx --yes https://infer.flow7.org/downloads/infer-setup-0.2.0.tgz setup --client litellm
npx --yes https://infer.flow7.org/downloads/infer-setup-0.2.0.tgz guide litellm

Needs Node.js 20 or newer. Other clients and options.

Configuration

Choose the model and route.

The SDK call and the proxy entry use the same selector.

Reading Infer’s public catalog…
SDK call and proxy entryPython · infer.yaml
Waiting for the live Infer catalog…

The proxy reads the key with os.environ/INFER_API_KEY; add model_info with the route's limits if your proxy needs them.

Good to know

Settings that matter.

  1. 01

    Nothing after /v1.

    LiteLLM appends the endpoint path itself.

  2. 02

    A stray base URL wins.

    LiteLLM reads OPENAI_BASE_URL before OPENAI_API_BASE, so pass api_base explicitly.

  3. 03

    Claude through anthropic/.

    LiteLLM's anthropic/ route appends /v1/messages, so its base is https://infer.flow7.org without /v1.

SourcesChecked against LiteLLM’s documentation and source on October 26, 2026. Infer has not recorded a live check with LiteLLM yet.
OpenAI-compatible endpoints ↗Proxy config ↗

Start with a capped key.

Create an account, add credit, and give the API key a spend limit before the first session.

Create accountSee models and prices