Configure models on the server

A self-hosted server runs the model families you give it credentials for. The shipped catalog covers five providers. An operator catalog adds your own families, models, prices, and OpenAI-compatible endpoints, such as vLLM, Ollama, or a gateway.

modelEnvironment in packages/agents/src/model-environment.ts parses this configuration. The API catalog, saved-agent chat, tasks, Slack agents, and plain conversations all use it. The api and worker read it once at startup, so changing it is a restart. Invalid configuration stops them at startup with Invalid server model configuration or Invalid model catalog. The errors name the variable, family, or model but never include values.

Workspace admins then choose which of the server’s families their workspace uses. See Models.

Providers

Set the API key of at least one provider. Each family in the catalog runs on one provider: the provider of its current model when that key is set, otherwise the next catalog entry whose provider has a key.

Provider	Credentials	Families it serves	Role defaults
Anthropic	ANTHROPIC_API_KEY	claude-haiku, claude-sonnet, claude-opus, claude-fable	Sonnet for chat, review, and plans; Haiku for titles, compaction, and the Gmail heartbeat
OpenAI	OPENAI_API_KEY	gpt-luna, gpt-sol, gpt-astra	Sol for chat, review, and plans; Luna for titles and the heartbeat
xAI	XAI_API_KEY	grok, grok-fast	Grok Fast for titles, compaction, and the heartbeat
Google Gemini	GEMINI_API_KEY	gemini-flash, gemini-pro	None
Azure Foundry	AZURE_FOUNDRY_ENDPOINT and AZURE_FOUNDRY_API_KEY	gpt-luna, gpt-sol, gpt-astra, deepseek-pro, deepseek-flash, grok, grok-fast, kimi, kimi-code	Every role, including the public guide

The shipped catalog is packages/config/src/model-families.json (families) and packages/config/src/model-pricing.json (models and prices). A model’s providerModelId is the name sent to the provider. Azure deployments must use exactly that name, for example gpt-6-sol or DeepSeek-V4-Pro.

AZURE_FOUNDRY_ENDPOINT is your Foundry resource origin with a trailing slash, for example https://<account>.services.ai.azure.com/. With both Azure and OpenAI or xAI keys, GPT and Grok families run on Azure, because the shipped catalog lists their Azure models first.

Choose families

Variable	Purpose
MODEL_FAMILIES	Comma-separated family IDs to enable. Unset enables every family a configured key can serve. A listed family without a credentialed provider fails startup.
MODEL_DEFAULT_FAMILY	Optional. Family for saved agents that have none, and for agent-instruction improvements. Must be enabled. Defaults to the default-chat role.

List your families

Role defaults choose among the families named in MODEL_FAMILIES. With it unset, every credentialed family is still enabled, but no family fills a role: new chats use the shipped default claude-sonnet, chats get no titles, and skill review and the plan specialist have no model. Set MODEL_FAMILIES, or set the role variables below, until this is fixed.

A minimal Anthropic server:

ANTHROPIC_API_KEY=sk-ant-...
MODEL_FAMILIES=claude-haiku,claude-sonnet,claude-opus

The other providers follow the same pattern:

# OpenAI
OPENAI_API_KEY=sk-...
MODEL_FAMILIES=gpt-luna,gpt-sol,gpt-astra

# xAI
XAI_API_KEY=xai-...
MODEL_FAMILIES=grok,grok-fast

# Google Gemini
GEMINI_API_KEY=...
MODEL_FAMILIES=gemini-flash,gemini-pro

# Azure Foundry
AZURE_FOUNDRY_ENDPOINT=https://<account>.services.ai.azure.com/
AZURE_FOUNDRY_API_KEY=...
MODEL_FAMILIES=gpt-luna,gpt-sol,deepseek-flash,grok,kimi

xAI alone has no default-chat family and Gemini alone has no roles at all, so those features fall back to the default chat family (see Roles). Startup logs a warning for each fallback.

Gateways and base URLs

To send a provider’s traffic through a gateway (LiteLLM, Portkey, a proxy) or a regional endpoint, replace its base URL. Hosts other than the built-in APIs must also be listed in MODEL_ENDPOINT_HOSTS.

Variable	Default	Notes
ANTHROPIC_BASE_URL	https://api.anthropic.com	/v1/messages is appended; leave out /v1.
OPENAI_BASE_URL	https://api.openai.com/v1	A URL without a path gets /v1. A URL with a path is the API base /chat/completions and /responses are appended to.
XAI_BASE_URL	https://api.x.ai/v1	Same path rule as OpenAI.
GEMINI_BASE_URL	https://generativelanguage.googleapis.com	/v1beta/models/... is appended; leave out /v1beta.

Operator catalog

The operator catalog adds families and models to the shipped catalog, or replaces it. It is JSON in the shape of the shipped files:

{ "families": [], "models": [] }

Variable	Purpose
MODEL_CATALOG_FILE	Path to the catalog JSON inside the api and worker containers, for example /run/tulipfarm-config/model-catalog.json.
MODEL_CATALOG	The same JSON inline. Set this or MODEL_CATALOG_FILE, not both. In .env, single-quote it.
MODEL_CATALOG_MODE	extend (default) or replace. Requires one of the two above.

Extend adds your entries to the shipped catalog. An entry with a shipped ID replaces that entry whole: copy the shipped entry and change what you need. Shipped models that no family lists any more are dropped. Replace uses only your entries. In both modes, MODEL_FAMILIES still chooses which families are enabled.

Full example

This catalog extends the shipped one with a free self-hosted Llama on vLLM and a metered Qwen model on a gateway. Both use the openai-compatible provider.

model-catalog.json

{
  "families": [
    {
      "id": "local-llama",
      "name": "Llama (local)",
      "vendor": "meta",
      "currentModelId": "llama-3-3-70b-local",
      "modelIds": ["llama-3-3-70b-local"],
      "pricing": "unmetered",
      "roles": ["title", "compaction", "heartbeat"]
    },
    {
      "id": "qwen-gateway",
      "name": "Qwen Coder",
      "vendor": "alibaba",
      "currentModelId": "qwen3-coder-gateway",
      "modelIds": ["qwen3-coder-gateway"],
      "capabilities": { "api": "chat-completions", "toolChoice": "auto" }
    }
  ],
  "models": [
    {
      "id": "llama-3-3-70b-local",
      "name": "Llama 3.3 70B (vLLM)",
      "provider": "openai-compatible",
      "providerModelId": "meta-llama/Llama-3.3-70B-Instruct",
      "endpoint": "https://llm.internal.example.com/v1",
      "credentialEnv": "LOCAL_LLM_API_KEY",
      "metadata": { "max_input_tokens": 128000, "max_output_tokens": 8192 }
    },
    {
      "id": "qwen3-coder-gateway",
      "name": "Qwen3 Coder (gateway)",
      "provider": "openai-compatible",
      "providerModelId": "qwen/qwen3-coder",
      "endpoint": "https://gateway.example.com/v1",
      "credentialEnv": "GATEWAY_API_KEY",
      "metadata": { "max_input_tokens": 262144, "max_output_tokens": 65536 },
      "rates": [
        {
          "validFrom": null,
          "validUntilExclusive": null,
          "promptTokensAbove": null,
          "usdPerMillionTokens": { "input": 0.4, "output": 1.6, "cacheRead": 0.04 }
        },
        {
          "validFrom": null,
          "validUntilExclusive": null,
          "promptTokensAbove": 128000,
          "usdPerMillionTokens": { "input": 0.8, "output": 3.2, "cacheRead": 0.08 }
        }
      ]
    }
  ]
}

Family fields

Field	Required	Meaning
id	Yes	Lowercase letters, digits, and hyphens, starting with a letter; up to 100 characters. This is the ID in MODEL_FAMILIES.
name	Yes	Display name.
vendor	Yes	Who makes the model, such as meta. Lowercase letters, digits, ., _, -.
currentModelId	Yes	The model the family runs. Must be in modelIds.
modelIds	Yes	Every model of the family, at least one. A model belongs to exactly one family.
capabilities	No	Wire behavior; see Capabilities.
roles	No	Features the family is a default for; see Roles.
pricing	No	"unmetered" for models without prices; see Prices.

Model fields

Field	Required	Meaning
id	Yes	Same format as a family ID.
name	Yes	Display name.
provider	Yes	anthropic, openai, xai, google, azure, or openai-compatible.
providerModelId	Yes	The model or deployment name sent to the provider, up to 200 characters.
metadata	Yes	Must include max_input_tokens and max_output_tokens.
rates	Unless unmetered	Prices; see Prices. Not allowed in an unmetered family.
endpoint	openai-compatible only	API base URL.
credentialEnv	openai-compatible only	Name of the environment variable holding the key.
metadataKey, pricingSource, pricingScope, notes	No	Bookkeeping, as in the shipped file.

Unknown fields are rejected. So are duplicate IDs, a currentModelId outside modelIds, a listed model that does not exist, a model in no family or in two, and a catalog with no families. The error names the entry:

Invalid model catalog (MODEL_CATALOG_FILE=/run/tulipfarm-config/model-catalog.json): family "qwen-gateway": model "qwen3-coder-gateway" has no rates; add rates or set "pricing": "unmetered" on the family

Mount the catalog in Docker Compose

The bundled self-host/compose.yaml forwards only the variables it knows. Add a compose.override.yaml next to it to mount the catalog and pass your credentialEnv keys and any role variables to the api and worker:

self-host/compose.override.yaml

services:
  api:
    volumes:
      - ./model-catalog.json:/run/tulipfarm-config/model-catalog.json:ro
    environment:
      LOCAL_LLM_API_KEY: ${LOCAL_LLM_API_KEY:?}
      GATEWAY_API_KEY: ${GATEWAY_API_KEY:?}
  worker:
    volumes:
      - ./model-catalog.json:/run/tulipfarm-config/model-catalog.json:ro
    environment:
      LOCAL_LLM_API_KEY: ${LOCAL_LLM_API_KEY:?}
      GATEWAY_API_KEY: ${GATEWAY_API_KEY:?}
      CHAT_TITLE_MODEL_FAMILY: ${CHAT_TITLE_MODEL_FAMILY:-}

Then set, in self-host/.env or your secret store:

ANTHROPIC_API_KEY=sk-ant-...
MODEL_FAMILIES=claude-haiku,claude-sonnet,claude-opus,local-llama,qwen-gateway
MODEL_CATALOG_FILE=/run/tulipfarm-config/model-catalog.json
MODEL_ENDPOINT_HOSTS=llm.internal.example.com,gateway.example.com
LOCAL_LLM_API_KEY=...
GATEWAY_API_KEY=...
CHAT_TITLE_MODEL_FAMILY=local-llama

Run docker compose up -d again to restart the api and worker with the new catalog.

OpenAI-compatible models

A model with "provider": "openai-compatible" runs on any API that speaks OpenAI chat completions, such as vLLM, Ollama, LM Studio, or OpenRouter.

• endpoint is the API base URL, with the same path rule as OPENAI_BASE_URL: https://llm.example.com becomes https://llm.example.com/v1/, and https://gateway.example.com/openai is used as given.
• credentialEnv names the variable that holds the key. It must end in _API_KEY, use capital letters, digits, and underscores, and not start with VITE_, NEXT_PUBLIC_, or PUBLIC_. The family turns on, like any other, only when that variable is set and not empty. For a server that needs no key, such as Ollama, set any value.
• Its host must be in MODEL_ENDPOINT_HOSTS, even localhost.

Only openai-compatible models take endpoint and credentialEnv; they are required there and rejected on any other provider.

Endpoint allowlist

A model credential is only sent to an allowed endpoint. The api and worker check every endpoint at startup and again before each request.

• The built-in APIs (api.anthropic.com, api.openai.com, api.x.ai, generativelanguage.googleapis.com) and Azure Foundry resource hosts (*.openai.azure.com, *.cognitiveservices.azure.com, *.services.ai.azure.com) are always allowed.
• Every other host must be listed in MODEL_ENDPOINT_HOSTS: comma-separated host names, matched exactly. No scheme, port, path, or wildcards.
• Endpoints use https. http is allowed only for localhost, 127.0.0.1, and ::1, and never when NODE_ENV=production.
• URLs must not contain credentials, a query, or a fragment.

Invalid server model configuration: model "llama-3-3-70b-local" endpoint host llm.internal.example.com is not in MODEL_ENDPOINT_HOSTS

Run a local model

For development outside Docker, this catalog replaces the shipped one with a single Ollama model that fills every role except the public guide:

model-catalog.json

{
  "families": [
    {
      "id": "local-llama",
      "name": "Llama (local)",
      "vendor": "meta",
      "currentModelId": "llama-3-3-70b-local",
      "modelIds": ["llama-3-3-70b-local"],
      "pricing": "unmetered",
      "roles": ["default-chat", "title", "compaction", "review", "plan", "heartbeat"]
    }
  ],
  "models": [
    {
      "id": "llama-3-3-70b-local",
      "name": "Llama 3.3 70B (Ollama)",
      "provider": "openai-compatible",
      "providerModelId": "llama3.3:70b",
      "endpoint": "http://localhost:11434/v1",
      "credentialEnv": "OLLAMA_API_KEY",
      "metadata": { "max_input_tokens": 128000, "max_output_tokens": 8192 }
    }
  ]
}

MODEL_CATALOG_FILE=./model-catalog.json
MODEL_CATALOG_MODE=replace
MODEL_FAMILIES=local-llama
MODEL_ENDPOINT_HOSTS=localhost
OLLAMA_API_KEY=ollama

The bundled Docker images set NODE_ENV=production, so http endpoints are rejected there. In Compose, serve the model over https on a host you list in MODEL_ENDPOINT_HOSTS, for example behind a reverse proxy with an internal certificate. localhost inside a container is the container itself, not the Docker host.

The model must handle function tools: only enable a family after it answers a tool-calling request with your key.

Capabilities

capabilities on a family describe how TulipFarm talks to its models. Each field defaults per provider; the family overrides those defaults.

Field	Default	Meaning
api	chat-completions for openai, xai, azure, openai-compatible; anthropic-messages for anthropic; gemini for google	Wire API. OpenAI-wire providers also accept responses. Other combinations are rejected.
toolChoice	forced	forced requires a tool call when tools are offered; auto lets the model answer in text.
reasoningOffForTools	false	Turn reasoning off on requests that offer function tools.
reasoningTokensInOutput	false	Count the provider’s separate reasoning-token counter as output.
streaming	true	Stream chat replies.
responseModelAliases	[]	Other model names the provider may report in responses, besides providerModelId and its -YYYY-MM-DD snapshots.

The shipped families use these: Claude Sonnet, Opus, and Fable set toolChoice: auto; Haiku accepts claude-haiku-4-5-20251001 in responses; GPT Luna and Sol turn reasoning off for tools; GPT Astra uses responses; Grok counts reasoning tokens as output. Overriding a shipped family replaces its capabilities with yours.

Roles

Features that run without a chosen model use a role’s family. A family’s roles list makes it a candidate. Each role takes the first family in catalog order that has the role and is in MODEL_FAMILIES; operator families come after shipped ones in extend mode. A variable overrides the catalog.

Role	Used for	Override
default-chat	New chats and the web model picker’s default	None; see MODEL_DEFAULT_FAMILY for agents
title	Chat titles	CHAT_TITLE_MODEL_FAMILY (worker)
compaction	Conversation summaries	Workspace compaction policy
review	Skill safety review	SKILL_REVIEW_MODEL_FAMILY (worker)
plan	The plan specialist	PLAN_SPECIALIST_MODEL_FAMILY (worker)
heartbeat	Gmail triage	GMAIL_HEARTBEAT_MODEL_FAMILY (worker)
public-guide	The public TulipFarm guide	PUBLIC_GUIDE_MODEL_FAMILY (api)

• When no enabled family has a role, the feature uses the default-chat family, and default-chat falls back to the first enabled family. Startup logs a warning such as No family in MODEL_FAMILIES has the catalog role "compaction".
• Titles have no fallback: the chat model titles its own chat.
• An override outside MODEL_FAMILIES leaves that feature without a model, and startup warns.
• The compaction families are the summary models a workspace may choose. New workspaces start with the compaction role’s family, automatic compaction on, a 70% trigger, and a 30% target (packages/config/src/compaction.ts).
• The public guide only works on an Azure route with the chat-completions API. Any other family makes the guide answer 503, with a startup warning. It also needs FEATURE_FLAG_PUBLIC_TULIPFARM_GUIDE.

Two other features take a family only by variable: TABLE_SEARCH_MODEL_FAMILY (table search translation; unset disables it) and RECOMMENDATION_MODEL_FAMILY (recommendations; unset disables them).

Prices and credits

Every metered call is costed from the effective catalog. A model’s rates hold USD per million tokens:

{
  "validFrom": null,
  "validUntilExclusive": null,
  "promptTokensAbove": null,
  "usdPerMillionTokens": {
    "input": 2,
    "output": 10,
    "cacheRead": 0.2,
    "cacheWrite5m": 2.5,
    "cacheWrite1h": 4
  }
}

• input and output are required. cacheRead, cacheWrite, cacheWrite5m, and cacheWrite1h are needed only if the provider reports those tokens; a call that uses an unpriced category fails rather than costing zero.
• At most six decimal places.
• One rate must have promptTokensAbove: null, the base price. A rate with promptTokensAbove: 128000 applies to the whole request once its prompt, including cached tokens, is above 128,000 tokens; the highest matching tier wins.
• validFrom and validUntilExclusive are ISO dates or null; validFrom must be before validUntilExclusive.

Contract prices

To use your negotiated prices for a shipped model, extend the catalog with that model’s entry under the same ID and your rates:

model-catalog.json

{
  "models": [
    {
      "id": "claude-sonnet-5",
      "name": "Claude Sonnet 5",
      "provider": "anthropic",
      "providerModelId": "claude-sonnet-5",
      "metadata": { "max_input_tokens": 1000000, "max_output_tokens": 128000 },
      "rates": [
        {
          "validFrom": null,
          "validUntilExclusive": null,
          "promptTokensAbove": null,
          "usdPerMillionTokens": {
            "input": 1.6,
            "output": 8,
            "cacheRead": 0.16,
            "cacheWrite5m": 2,
            "cacheWrite1h": 3.2
          }
        }
      ]
    }
  ]
}

The entry replaces the shipped one whole, so its rates are exactly yours.

Unmetered models

Set "pricing": "unmetered" on a family for models you don’t pay per token for, such as a model on your own GPUs, and leave rates out of its models. Tokens are still recorded, every call costs zero, and the web app shows “Not priced”. To make a shipped family unmetered, override the family and its models without rates.

Credits

Usage is recorded as provider cost. A self-hosted server converts it at one credit per US cent, with no markup: a model priced at 200 US cents per million input tokens shows 200 credits per million. Credits are unlimited by default; usage is still metered and shown on the Usage page. Extensions can replace the credit pricing and add a credit limit.

Catalog revisions

Each server records a mapping revision and a price revision. With the shipped catalog they are families-2026-10-19.1 and prices-2026-10-19.1. Any other catalog gets families- or prices- and a content hash. Families and prices hash separately, so new prices change only the price revision. A family’s capabilities and roles and a model’s endpoint and credentialEnv are not part of either hash.

Changing the configuration

New configuration takes effect when the api and worker restart.

• At startup, the api and worker record their configuration bundle in the database. Each model call records the bundle it started under.
• A call that started before the change is charged at the rates of its original bundle, even when the new catalog has different prices.
• Each attempt is checked against the current configuration before it runs. If its family was removed from MODEL_FAMILIES or its provider route is gone, the call fails instead of running on a model you turned off. Removing a model from the catalog also stops calls that started on it.
• New calls use the new configuration. A family that workspaces had enabled but the server no longer offers shows as unavailable.

What stays in the database

• Which families each workspace enables, and its fallback rules (tf_model_availability, tf_model_fallback_rules).
• The workspace compaction policy (tf_compaction_policy).
• Usage records.
• Route cooldowns after overload, timeout, or transport failures. The API stores them in app.model_route_health; each worker replica keeps its own in memory.

Knowledge embeddings

Enabling embeddings permits paid query requests and automatic embedding of eligible notes and promoted Files.

KNOWLEDGE_EMBEDDINGS_ENABLED=true

parseKnowledgeEmbeddingsConfig in packages/config/src/knowledge-embeddings.ts returns disabled when the flag is absent, empty, or false. Only exact true enables it, with a valid endpoint and a nonempty server key. Invalid values and public-prefixed embedding or Azure variables fail with Invalid server knowledge embeddings configuration, without values in the error. The embedding profile is separate from MODEL_FAMILIES and the model catalog; do not add it as a chat family.

Embedding providers

KNOWLEDGE_EMBEDDINGS_PROVIDER selects the transport.

Variable	Purpose
KNOWLEDGE_EMBEDDINGS_PROVIDER	azure-foundry (default) or openai-compatible.
AZURE_FOUNDRY_ENDPOINT	Azure provider only. Your Foundry origin; any HTTPS Foundry origin is accepted. Sent with an api-key header to openai/v1/embeddings.
AZURE_FOUNDRY_API_KEY	Azure provider key.
KNOWLEDGE_EMBEDDINGS_BASE_URL	OpenAI-compatible provider only, for example https://api.openai.com/v1. The adapter posts to <base URL>/embeddings with a bearer key. Requires HTTPS, except plain HTTP on localhost, 127.0.0.1, or [::1]. MODEL_ENDPOINT_HOSTS does not apply.
KNOWLEDGE_EMBEDDINGS_API_KEY	OpenAI-compatible provider key.
KNOWLEDGE_EMBEDDINGS_MODEL	Optional request model or deployment name. Defaults to text-embedding-3-large.
KNOWLEDGE_EMBEDDINGS_DIMENSIONS	Optional. Must be 1536 when set.

Every provider must serve the stored profile: responses must report model text-embedding-3-large with 1,536 dimensions. Workspace storage declares embedding vector(1536), and vectors, cache entries, and published search state are keyed by the profile ID. Other models are rejected as invalid_model or invalid_vectors. Use KNOWLEDGE_EMBEDDINGS_MODEL only when a deployment or gateway names that model differently. A different model or dimension needs a new profile and migration, not configuration. Token limits use cl100k_base whichever provider serves the request. If no provider can serve that profile, keep embeddings disabled.

Provider variables change only the transport, not the profile.