DOCS — PRODUCT · PROVIDERS

Providers¶

A provider is the service that runs the language model driving the agent. OpenMed supports four: OpenAI (via an API key or a ChatGPT OAuth sign-in), Anthropic, and OpenRouter. Everything else in OpenMed — the medical tools, workflows, and audit trail — behaves identically regardless of which one you choose.

This page covers credentials, model selection, and the behaviour specific to each provider. For runtime settings that are not provider-related, see Configuration.

Choosing a provider¶

Provider	Auth	Use it for
OpenAI	ChatGPT OAuth (`openmed login`) or `OPENAI_API_KEY`	The default path. GPT-5.5 and GPT-5.6 models.
Anthropic	`ANTHROPIC_API_KEY`	Claude models, with prompt caching enabled by default.
OpenRouter	`OPENROUTER_API_KEY`	Models OpenMed does not host itself — Gemini, Kimi, GLM, Qwen, Llama and several hundred more, on one key.
Custom	optional key	One OpenAI-compatible server you operate — vLLM, SGLang, llama.cpp, Ollama, LM Studio, or a gateway.

Provider keys are never mixed: an OpenRouter key is only ever sent to OpenRouter, and an OpenAI or Anthropic key is never sent there.

You can configure more than one at a time. Which provider handles a request is decided by the model you select — see How a provider is selected.

Where credentials live¶

Path	Holds
`~/.openmed/auth.json`	ChatGPT OAuth tokens, written by `openmed login`
`~/.openmed/provider_secrets.json`	OpenAI / Anthropic / OpenRouter API keys, and the OpenRouter model list
`~/.openmed/settings.json`	Runtime behaviour such as permission mode — not secrets

Store keys without hand-editing JSON:

openmed config provider-set openai --api-key "sk-..."
openmed config provider-set anthropic --api-key "sk-ant-..."
openmed config provider-set openrouter --api-key "sk-or-..."
openmed config provider-show

provider-show reports which providers are configured without printing secret values. To remove a stored key, use --unset-api-key with the same command.

Editing the file directly¶

~/.openmed/provider_secrets.json is written with 0600 permissions. Values may be literal strings, or {env:VAR_NAME} references that resolve at load time — useful when the file is synced between machines and you would rather not store the secret in it:

{
  "provider": {
    "openai": {
      "options": {
        "apiKey": "{env:OPENAI_API_KEY}"
      }
    },
    "anthropic": {
      "options": {
        "apiKey": "{env:ANTHROPIC_API_KEY}"
      }
    },
    "openrouter": {
      "options": {
        "apiKey": "sk-or-v1-...",
        "models": ["qwen/qwen3.8-27b", "z-ai/glm-5.2"]
      }
    },
    "custom": {
      "options": {
        "endpoints": [
          {"baseURL": "http://localhost:8000/v1", "model": "Qwen/Qwen3.8-27B", "apiKey": "{env:MY_SERVER_KEY}"}
        ]
      }
    }
  }
}

Only the custom section carries endpoint URLs (inside its endpoints list) — hosted provider endpoints are fixed, and a stored baseURL on them is ignored and stripped.

One JSON object, one provider map

Every provider is a key inside the single provider object above. A common mistake is to paste a second example after the first, which produces two objects and makes the whole file invalid — at which point all your keys are ignored and every provider shows as not configured. If that happens, openmed config provider-show now names the file and the line.

A flatter shape is accepted as an alternative to the block above — use one or the other, never both in the same file:

{
  "openrouter": {
    "api_key": "{env:OPENROUTER_API_KEY}",
    "models": ["qwen/qwen3.8-27b", "x-ai/grok-4.6"]
  }
}

Check the file is valid at any time:

openmed config provider-show

It prints which providers are configured, and exits non-zero with the parse error if the file is malformed.

Precedence is explicit runtime config → environment variable → this file → default. Provider config is read once per process, so restart OpenMed after editing the file by hand.

Provider base URLs for OpenAI and Anthropic are intentionally not part of this config surface during preview; OpenMed controls those endpoints internally. OpenRouter is the supported path for running models OpenMed does not host itself.

OpenAI¶

The default provider. Two independent auth paths — OAuth, or an API key.

Uses your ChatGPT subscription rather than API billing:

openmed login             # writes ~/.openmed/auth.json
openmed oauth status
openmed oauth logout

Credentials are reused until they expire. If auth status reports the token is no longer valid, openmed login -f refreshes the single global auth file. openmed login --no-browser prints the authorization URL instead of opening a browser, and --timeout <seconds> extends the default five-minute wait.

An interactive sign-in can only be started by openmed login — never implicitly from inside the TUI. If you sign in from another terminal while the TUI is open, your next message picks up the new credentials without a restart.

API key¶

openmed config provider-set openai --api-key "sk-..."
# or
export OPENAI_API_KEY=sk-...

No openmed login step is required when using a key.

Models¶

gpt-5.6-sol, gpt-5.6-terra (default), gpt-5.6-luna, and gpt-5.5. gpt-5.6 is an alias for gpt-5.6-terra. When both OAuth and an API key are present, these models use OAuth first and fall back to the key.

Anthropic¶

openmed config provider-set anthropic --api-key "sk-ant-..."
# or
export ANTHROPIC_API_KEY=sk-ant-...

openmed agent --provider anthropic --model claude-sonnet-4-6

Anthropic has no OAuth path — an API key is the only option.

Models¶

claude-opus-4-8, claude-opus-4-7, claude-sonnet-4-6 (the Anthropic default), and claude-haiku-4-5-20251001. Selecting any claude-* model routes to Anthropic when Anthropic credentials are available.

Prompt caching¶

Enabled by default. OpenMed marks the stable tool schema block and the system prompt as 5-minute cache breakpoints, and enables Anthropic's top-level automatic cache control for the growing conversation, so long tool runs do not resend the entire prior transcript as fresh input every turn.

Variable	Purpose	Default
`OPENMED_ANTHROPIC_PROMPT_CACHE`	Enable caching for stable tool/system prefixes	`1`
`OPENMED_ANTHROPIC_PROMPT_CACHE_TTL`	Cache TTL — `5m` or `1h`	`5m`
`OPENMED_ANTHROPIC_AUTOMATIC_CACHE`	Top-level automatic caching for multi-turn conversations	`1`

Rate-limit guard¶

OpenMed also guards Claude's input-token-per-minute limits locally. Cache reads are excluded from Anthropic's rate-limit accounting, but cache writes and uncached conversation growth still count. The Anthropic SDK tracks uncached input usage, paces requests near the local budget, compacts stale history when a cache miss would make a request too large, and retries transient 429 responses using Claude's retry-after header.

Variable	Purpose	Default
`OPENMED_ANTHROPIC_RATE_LIMIT_GUARD`	Pace and compact before ITPM limits are exceeded	`1`
`OPENMED_ANTHROPIC_ITPM_LIMIT`	Override the input-token-per-minute limit	model default
`OPENMED_ANTHROPIC_ITPM_SAFETY_RATIO`	Fraction of the limit to use before pacing	`0.85`
`OPENMED_ANTHROPIC_RATE_LIMIT_RETRIES`	Automatic retries after a `429`	`2`

OpenRouter¶

OpenRouter gives you models OpenMed does not host itself — Gemini, Kimi, GLM, Qwen, Llama and several hundred others — through one API key. It is a separate provider: OpenRouter credentials are never sent to OpenAI or Anthropic, and an OpenAI key is never sent to OpenRouter.

export OPENROUTER_API_KEY="sk-or-..."
openmed agent --provider openrouter

Choosing which models appear¶

Configure a short list rather than all several hundred. Those models appear as an OpenRouter group below Anthropic and OpenAI in the /model picker:

openmed config provider-set openrouter \
  --api-key "sk-or-..." \
  --models "google/gemini-3.7-flash,qwen/qwen3.8-27b,z-ai/glm-5.2"

Or in ~/.openmed/provider_secrets.json directly (as one key inside the single provider object — see Editing the file directly):

{
  "provider": {
    "openrouter": {
      "options": {
        "apiKey": "sk-or-v1-...",
        "models": [
          "google/gemini-3.7-flash",
          "qwen/qwen3.8-27b",
          "z-ai/glm-5.2"
        ]
      }
    }
  }
}

qwen/qwen3.8-27b is OpenMed's default OpenRouter model. Gemini is shown here because people ask for it, but see the warning below before using it for clinical work — Google's API terms prohibit that use.

The list is stored as models in the openrouter section of ~/.openmed/provider_secrets.json — see Editing the file directly. OPENROUTER_MODELS (comma-separated) overrides the stored list for one session, and openmed config provider-set openrouter --unset-models removes it. With nothing configured, OpenMed uses qwen/qwen3.8-27b.

Browse what OpenRouter offers:

openmed config openrouter-models --search gemini
openmed config openrouter-models --all      # include models OpenMed cannot use
openmed config openrouter-models --refresh  # bypass the 24h metadata cache

OpenRouter environment variables¶

Variable	Purpose	Default
`OPENROUTER_API_KEY`	The OpenRouter key. Required — OpenMed never falls back to another provider's key.	unset
`OPENROUTER_MODELS`	Comma-separated model ids offered in `/model`; overrides the stored list	stored list, else `qwen/qwen3.8-27b`
`OPENMED_OPENROUTER_MODELS_TTL_SECONDS`	Lifetime of cached model metadata under `~/.openmed/cache/`	`86400` (24h)

The OpenRouter base URL is fixed to https://openrouter.ai/api/v1. For a self-hosted or gateway server, use the custom provider.

Recommended models¶

These are the models OpenMed has actually run through its own agent loop with the full 93-tool schema, cross-checked against OpenRouter's τ²-Bench Airline (multi-turn tool use) and GPQA Diamond (reasoning) leaderboards.

Open-weight — self-hostable, and the starting point for most users:

openmed config provider-set openrouter --models "\
qwen/qwen3.8-27b,\
z-ai/glm-5.2,\
qwen/qwen3.5-397b-a17b,\
nvidia/nemotron-3-ultra-550b-a55b,\
xiaomi/mimo-v2.5-pro,\
qwen/qwen3.5-122b-a10b,\
minimax/minimax-m3,\
moonshotai/kimi-k3,\
qwen/qwen3.8-2.4t-a95b,\
deepseek/deepseek-v4-flash-0731,\
deepseek/deepseek-v4-pro-0813"

Model	Licence	Notes
`qwen/qwen3.8-27b`	Apache-2.0	Best value: τ²-Bench #3 overall, 1M context, vision
`z-ai/glm-5.2`	MIT	Best all-round balance; low hallucination rate
`qwen/qwen3.5-397b-a17b`	open	Strong on both tool use and reasoning
`nvidia/nemotron-3-ultra-550b-a55b`	permissive	Fastest model tested
`xiaomi/mimo-v2.5-pro`	open	Very well calibrated; 1M context
`qwen/qwen3.5-122b-a10b`	open	Outstanding cost/capability, under a cent per turn
`minimax/minimax-m3`	open	Best-calibrated model evaluated; GPQA top-10
`moonshotai/kimi-k3`	custom	Strongest reasoning of any open model
`qwen/qwen3.8-2.4t-a95b`	bespoke	Highest agentic score; licence is not permissive
`deepseek/deepseek-v4-flash-0731`	MIT	Cheapest model listed ($0.0018/turn), 1M context — but see the DeepSeek warning below
`deepseek/deepseek-v4-pro-0813`	MIT	τ²-Bench #8, GPQA #21 — verify every code it emits; see the DeepSeek warning below

Closed, hosted-only:

Model	Notes
`x-ai/grok-4.6`	Best-placed non-OpenAI/Anthropic model on healthcare benchmarks
`qwen/qwen3.8-max`	Top agentic tier; rarely refuses clinical content
`amazon/nova-pro-v1`	Fastest closed model tested
`qwen/qwen3.7-max`	Best published clinical benchmark results of the closed set
`x-ai/grok-4.5`	Same posture as 4.6, roughly twice as fast

Budget tiers that also passed cleanly: upstage/solar-pro4, amazon/nova-lite-v1, google/gemma-4-26b-a4b-it, qwen/qwen3.6-35b-a3b, nvidia/nemotron-3.5-lightning.

Fast, well-calibrated additions from a volume campaign (2026-08-31). These are the base models, worth running free or paid:

Model	Notes
`cohere/north-mini-code`	Fastest clean probe of any model tested (~10s), every clean run perfect
`inclusionai/ling-3.0-flash-fin`	Every clean run perfect, ~11s. Not `ling-3.0-flash`, which is excluded for hallucination — this is the `-fin` variant
`minimax/minimax-m2.7`	The reliable long-horizon pick. Slower (~21s)

Trying OpenMed for free¶

Several of the models above are also served on a :free tier, so you can drive the whole 93-tool agent without spending anything. The numbers below come from 5,500+ live runs of OpenMed's own agent loop against the free tiers, not from a leaderboard.

Start with one line:

openmed config provider-set openrouter --api-key "sk-or-..." \
  --models "minimax/minimax-m2.7:free,cohere/north-mini-code:free,inclusionai/ling-3.0-flash-fin:free"
openmed agent --provider openrouter --model "minimax/minimax-m2.7:free"

Free model	Clean runs	Perfect	Honest	Median	Long tasks
`minimax/minimax-m2.7:free`	2320/2327	92%	100%	21s	281/310
`minimax/minimax-m3:free`	1580/1620	72%	100%	18s	154/163
`cohere/north-mini-code:free`	273/273	100%	100%	10s	see note
`inclusionai/ling-3.0-flash-fin:free`	271/271	100%	100%	11s	see note

minimax/minimax-m2.7:free is the one to start with. It is the only free model that reliably finishes long, multi-step clinical work — 281 of 310 sustained 10-case runs — while staying honest on every single clean run.

cohere/north-mini-code:free and inclusionai/ling-3.0-flash-fin:free are the fastest and were flawless on every clean short probe. Their long-horizon column is marked "see note" deliberately: their sustained runs did not complete in our testing, but they were cut short by the daily request cap, not by any observed failure of the model. Treat them as proven for short tasks and unproven for long ones.

Not honest, not calibrated — the same thing across every one of these: no free model fabricated a file's contents or a clinical fact in any clean run. That property is why these are listed at all; see the warning below about choosing on benchmark rank.

nvidia/nemotron-3-ultra-550b-a55b:free — use the paid tier

The free tier of this model was usable in only 23 of 386 attempts, with a median of 63s, because its free capacity routes through a single congested upstream. The paid variant of the identical model scored 80/80 perfect and completed 12/12 long runs at 20s. This is a serving problem, not a model problem — but on the free tier it will waste your daily allowance.

What the free tier actually gives you

OpenRouter allows 1,000 free-model requests per day once an account has purchased at least $10 in credits, and 50 per day before that. Free models cost nothing to run; the credit only raises the request ceiling.

The allowance is a single pool shared across every free model, not a per-model quota — measured directly: once it is spent, models the account has never called return remaining=0 with the same reset timestamp as the one that drained it. Adding more free models to your config therefore diversifies what you can run, but does not increase your daily budget. It resets at 00:00 UTC.

A long agent run can use 50-80 requests, so long-horizon work exhausts the pool far faster than short tasks.

One exception worth knowing: a few :free models are served by providers with their own free capacity and do not draw on that shared pool — minimax/minimax-m2.7:free and minimax/minimax-m3:free both keep serving after it is exhausted. If you only configure one free model, make it one of those.

Read before choosing on benchmark rank alone

Some of the highest-ranked and most-used models on OpenRouter are not suitable for clinical work, and OpenMed's own testing is why. A high leaderboard rank measures tool mechanics and reasoning, not calibration — a model can drive the whole 93-tool loop flawlessly and still state a confident, wrong clinical fact. Several providers' terms also forbid medical use outright — Google's Gemini API terms prohibit use "in clinical practice, to provide medical advice"; Z.AI's and Tencent's terms carry comparable restrictions, and Tencent's licence excludes the EU, UK and South Korea entirely. Moonshot's hosted API forbids processing HIPAA-defined PHI, so Kimi models should be self-hosted for patient data. A BAA with an upstream provider does not survive routing through OpenRouter.

DeepSeek V4: listed, but check its output

deepseek/deepseek-v4-flash-0731 and deepseek/deepseek-v4-pro-0813 are listed above for their licence (MIT), their cost and their leaderboard placings — pro-0813 is τ²-Bench #8 and GPQA #21, and flash-0731 is the cheapest entry on the list at roughly $0.0018 per turn.

They are listed despite the DeepSeek family's published AA-Omniscience hallucination rates of 91.7–94.1%, the highest of any model OpenMed recommends. Both pass our tool-discipline probe — that is the point: the probe proves a model can hold the loop and not invent a file's contents, and it does not prove calibration.

In OpenMed's own repeated testing, pro-0813 returned ICD-10 R03.0 instead of I10 on four consecutive runs for a straightforward hypertension presentation, once describing the reading as "stage 2 hypertension" while coding elevated blood pressure without a diagnosis of hypertension — a self-contradiction. flash-0731 answered correctly every time.

If you need a DeepSeek model, prefer flash-0731. With either, treat generated codes, doses and clinical claims as drafts requiring review — which is what OpenMed's reviewer-gated workflow is for.

Mistral was evaluated on request and not added: mistral-medium-3-5 scores 100/100 on our live agent probes but carries an 81.6% published hallucination rate — roughly double the range of every model on this list. A clean probe proves tool mechanics, not calibration.

The full evidence table, including per-model probe results, hallucination rates and licence verification, is in PLANS/openrouter-recommended-models.md.

Not every OpenRouter model can run the agent¶

OpenRouter hosts image generators, audio models, and chat models without tool calling. OpenMed's tools are the product, so a model that cannot call tools cannot drive it. OpenMed reads each model's capabilities from OpenRouter's model metadata and refuses an incompatible model at startup with the reason, rather than failing part-way through a case:

OpenRouter model 'google/lyria-3-pro-preview' cannot drive the agent:
no tool-calling support (required by the agent).

Capabilities are also enforced per request:

Capability	Behavior
Tool calling	Required. Models without it are refused at startup.
Text output	Required. Image-only and audio-only models are refused.
Image input	Scanned documents and screenshots are sent only to vision models. On a text-only model the attachment is omitted and the message says so.
Context window	Taken from OpenRouter metadata, so compaction triggers at the model's real limit.
Reasoning effort	Clamped to the levels the model advertises, so `/model` never sends an effort the model rejects.
Max output tokens	Clamped to the model's provider cap.

The /model picker shows these as chips per model (vision · 1M ctx · reasoning), and marks a model incompatible before you select it.

Model quality is your call¶

OpenMed evaluates the curated OpenAI and Anthropic models it ships with. It does not evaluate the OpenRouter catalog. A model that passes the compatibility check can still be a poor fit for clinical work — weaker instruction-following, worse tool discipline, or a smaller effective context than its advertised window. Treat an OpenRouter model as unvalidated until you have evaluated it on your own cases. Every PHI, de-identification, and finalization gate behaves identically regardless of provider.

Prompts you send — including any PHI in them — go to OpenRouter and on to whichever upstream provider serves that model. Check OpenRouter's privacy settings for how your data is handled and whether prompt logging is enabled on your account.

Custom OpenAI-compatible endpoints¶

The custom provider points OpenMed at servers you operate — vLLM, SGLang, llama.cpp, Ollama, LM Studio, or a company gateway. Serving engines launch one model per endpoint, so the configuration is a list of endpoints, each with its own base URL, optional credential, and the one model it serves. Selecting a model routes the session to the endpoint that declares it. The hosted providers (OpenAI, Anthropic, OpenRouter) deliberately have no base-URL configuration — a provider means one thing, and any stored base URL for them is ignored and stripped.

Add one endpoint per model:

openmed config provider-set custom --base-url http://host1:8000/v1 --model "nvidia/Nemotron-3-Super-120B"
openmed config provider-set custom --base-url http://host2:8000/v1 --model "Qwen/Qwen3-235B-A22B" --api-key sk-host2
openmed config provider-set custom --base-url http://host4:8000/v1 --model "poolside/Laguna"

openmed config endpoint-doctor http://host1:8000/v1   # validate each FIRST
openmed agent --provider custom --model "poolside/Laguna"

Re-running with the same --model updates that endpoint (a stored key is kept unless you pass a new one); --remove-model <id> deletes it.

Or in ~/.openmed/provider_secrets.json directly — an endpoints list under the custom section:

{
  "provider": {
    "custom": {
      "options": {
        "endpoints": [
          {"baseURL": "http://host1:8000/v1", "model": "nvidia/Nemotron-3-Super-120B"},
          {"baseURL": "http://host2:8000/v1", "model": "Qwen/Qwen3-235B-A22B", "apiKey": "{env:HOST2_KEY}"},
          {"baseURL": "http://host4:8000/v1", "model": "poolside/Laguna"}
        ]
      }
    }
  }
}

apiKey is optional per endpoint (local servers are commonly keyless) and supports literal or {env:VAR} forms. The same one-object-one-provider-map rule from Editing the file directly applies.

For a quick test without touching the file, environment variables declare one additional endpoint: OPENMED_CUSTOM_BASE_URL, OPENMED_CUSTOM_MODEL, and optionally OPENMED_CUSTOM_API_KEY. They win over a stored endpoint that declares the same model.

What the provider does for you at startup:

Routes by model. Each session talks only to the endpoint declaring its model, with that endpoint's own credential — keys never cross hosts, and your OPENAI_API_KEY can never silently travel to your own server.
Consults the endpoint's /models listing. A model the server does not actually serve fails immediately, naming what it does serve.
Reads the served context window (vLLM/SGLang max_model_len) for automatic compaction. A server that reports none pins the window to unknown — automatic compaction is off for that session (see Context And Compaction) and a hosted catalog namesake can never supply a fabricated number.
Sends the safe request shape. Chat Completions only, temperature omitted, streamed usage requested, and the runtime guard refuses a turn whose "text" is really an unparsed tool call.
Refuses cleartext keys to remote hosts — plain http:// with a stored key is allowed only for localhost and private networks, enforced both in the CLI and again at resolution so a hand-edited file cannot bypass it.

Configured models appear as a Custom endpoint group in the /model picker, each row naming the host that serves it.

What custom models are not

OpenMed cannot vouch for a model it has never evaluated. Tool-calling discipline, clinical quality and safety behaviour are yours to validate — endpoint-doctor proves the plumbing, not the medicine. Per-model launch flags for the common engines, including the exact --tool-call-parser value (which fails silently when wrong), are in PLANS/model-serving-matrix.md.

Checking an endpoint before you trust it¶

openmed config endpoint-doctor tells you whether an OpenAI-compatible server — your own vLLM, SGLang, llama.cpp or Ollama, or a gateway — can actually drive the agent. It makes no changes and is safe to run against anything.

openmed config endpoint-doctor http://localhost:8000/v1

A healthy endpoint:

Endpoint: http://localhost:8000/v1
Model:    openmed-local

  [PASS] reachable: Connected to http://localhost:8000/v1.
  [PASS] auth: Credentials accepted.
  [PASS] models: 1 model(s) served: openmed-local
  [PASS] context: Context window 262,144 tokens.
  [PASS] streaming: Server streamed a response (SSE).
  [PASS] tools: Model emitted a streamed tool call.
  [PASS] tool_call_index: Tool-call deltas carry `index`.
  [PASS] temperature: Model accepts `temperature`.

Result: usable by OpenMed (2.4s)

Exit codes are 0 usable, 1 not usable, 2 bad input — so it works in CI. --json emits the same report as JSON. --model picks a specific model, and --quick sends one tool instead of OpenMed's full 93-tool schema (faster, but it will not surface large-tool-array failures).

What it checks, and why¶

Check	Catches
`reachable`	Wrong port, wrong scheme, DNS typo, TLS mismatch — each reported distinctly
`auth`	Missing or wrong API key (self-hosted servers 401; public catalogs do not)
`models`	Server up but serving nothing, or a listing that is not OpenAI-shaped
`context`	A window too small for OpenMed's ~25,300-token per-turn overhead
`streaming`	Gateways that buffer and break SSE — OpenMed only streams
`tools`	Tool calling disabled, unsupported, or rejected at 93 tools
`leaked_tool_call`	The safety check — a wrong tool-call parser returning tool syntax as prose
`tool_call_index`	Streamed deltas missing `index`, which breaks parallel calls
`finish_reason`	Shapes the runtime rejects, which would crash on the first tool turn
`temperature`	Reasoning models that reject the parameter

The failure that matters most¶

If the server's tool-call parser is missing or wrong, it returns the tool call as plain text with tool_calls empty and finish_reason: "stop". There is no error, and in a clinical agent that text is indistinguishable from real content. The doctor fails the endpoint and names the fix:

  [FAIL] leaked_tool_call: The model returned tool syntax as PLAIN TEXT
         (XML <tool_call> (Qwen 3.5+, GLM, MiniMax)) with no tool_calls
         (finish_reason='stop'). The server is not parsing tool calls.
          -> The tool-call parser is missing or wrong. vLLM needs BOTH
          -> --enable-auto-tool-choice AND --tool-call-parser <name> (there is no
          -> 'auto'); SGLang needs --tool-call-parser. Use the value for this model
          -> family — e.g. Qwen 3.5+ is `qwen3_coder`, NOT `hermes`.

It also catches the harder version of this, where a parser matches the first tool call and leaks the rest as text — the case that otherwise looks healthy.

Context: the quiet killer¶

OpenMed sends ~93 tool schemas (~21,000 tokens) plus a ~4,500-token system prompt on every turn. A window below that truncates every request:

  [FAIL] context: Context window is 16,384 tokens; OpenMed's tool schema alone
         is 20,791. Every request would be truncated.
          -> Raise the served context to at least 65,536: vLLM --max-model-len 65536,
          -> SGLang --context-length 65536, llama.cpp --ctx-size 65536 (its default is
          -> 4096), Ollama OLLAMA_CONTEXT_LENGTH=65536 before `ollama serve`.

llama.cpp defaults to 4096, Ollama auto-scales and can land at 4k, and Google's own vLLM recipe for Gemma 4 ships 16384 — all below the floor.

Serving your own model¶

Per-model launch flags, including the exact --tool-call-parser value for each family, are in PLANS/model-serving-matrix.md. The short version for the recommended single-GPU model:

vllm serve Qwen/Qwen3.8-27B \
  --served-model-name openmed-local \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --max-model-len 65536 \
  --host 0.0.0.0 --port 8000 --api-key sk-openmed-CHANGEME

openmed config endpoint-doctor http://localhost:8000/v1

Credentials

Prefer OPENMED_ENDPOINT_API_KEY over --api-key: arguments are visible in ps and shell history. The doctor refuses to send a key over cleartext http:// to a non-local host, strips any user:pass@ from the URL before printing anything, and never echoes the key into its output.

Web search¶

Web search is off by default on every provider, including OpenAI and ChatGPT OAuth, and is not a provider capability — it runs client-side so it behaves identically on Anthropic, OpenRouter or a local model. Enabling it sends search queries to a third-party vendor, so it is an explicit operator decision. See Web Search.

How a provider is selected¶

The TUI defaults to gpt-5.6-terra with medium reasoning. When multiple providers are configured, the model you select determines the provider:

claude-* models use Anthropic when Anthropic credentials are available.
The supported GPT-5.5 and GPT-5.6 models use ChatGPT OAuth first when ~/.openmed/auth.json exists, then fall back to the OpenAI key.
Namespaced model ids (vendor/model, such as google/gemini-3.7-flash) use OpenRouter.
With no credentials at all, the native fallback remains available.

Choose the model at launch, or at runtime with Ctrl+M or /model:

openmed agent --model gpt-5.6-terra --reasoning-effort medium
openmed agent --provider openrouter --model qwen/qwen3.8-27b

--provider forces a specific provider; omit it to let the model decide.

Reasoning effort¶

The picker exposes model-specific values: none, low, medium, high, xhigh, and max (GPT-5.6 and supported Anthropic models only). extra_high remains accepted as a compatibility alias and resolves to the highest compatible non-max OpenAI effort (xhigh).

For OpenRouter models the effort is clamped to the levels that model actually advertises, so a selection is never rejected by the provider.

Provider keys vs native medical services

OpenAI, Anthropic, and OpenRouter keys are model-provider credentials. OpenMed's native medical services — extraction, PII, de-identification, terminology, coding — are a separate service plane with their own configuration. See Native Medical Services.

Providers¶

Choosing a provider¶

Where credentials live¶

Editing the file directly¶

OpenAI¶

Sign in with ChatGPT (OAuth)¶

API key¶

Models¶

Anthropic¶

Models¶

Prompt caching¶

Rate-limit guard¶

OpenRouter¶

Choosing which models appear¶

OpenRouter environment variables¶

Recommended models¶

Trying OpenMed for free¶

Not every OpenRouter model can run the agent¶

Model quality is your call¶

Custom OpenAI-compatible endpoints¶

Checking an endpoint before you trust it¶

What it checks, and why¶

The failure that matters most¶

Context: the quiet killer¶

Serving your own model¶

Web search¶

How a provider is selected¶

Reasoning effort¶