AI models and gateway configuration

Four GTM roles can use different models independently:

Routing decision

If GTM_LLM_PROVIDER is omitted, the application chooses AI Gateway when DATABRICKS_HOST is present, an OpenAI-compatible endpoint when GTM_LLM_BASE_URL is present, and direct Anthropic otherwise.

Recommended: Databricks AI Gateway v2

GTM_LLM_PROVIDER=databricks-ai-gateway
DATABRICKS_HOST=https://adb-<workspace-id>.<shard>.azuredatabricks.net
GTM_JUDGE_MODEL=system.ai.claude-sonnet-4-6
GTM_DRAFT_MODEL=system.ai.claude-sonnet-4-6
GTM_CLASSIFIER_MODEL=system.ai.claude-haiku-4-5
GTM_REPLY_DRAFT_MODEL=system.ai.claude-haiku-4-5

Authentication uses the local Databricks CLI profile during development and the App service principal in production. The client sends the native Anthropic request shape to /ai-gateway/anthropic/v1; it does not emulate Anthropic through an OpenAI schema.

To inspect readiness:

DATABRICKS_HOST=<workspace-url> \
DATABRICKS_CONFIG_PROFILE=fabric-harness \
DATABRICKS_SQL_WAREHOUSE_ID=<warehouse-id> \
ENDPOINT_NAME=projects/<project>/branches/<branch>/endpoints/<endpoint> \
pnpm --filter @repo/gtm databricks:doctor

The doctor checks Gateway v2, discovers UC model services, performs a real Claude round-trip, reads the gateway usage table, validates MLflow/OTel/analytics, and opens a TLS Lakebase connection.

Governed bring-your-own provider

Use a Unity Catalog Model Provider Service when the organization wants its own Anthropic or Amazon Bedrock account while retaining Databricks governance.

GTM_LLM_PROVIDER=databricks-ai-gateway
GTM_DATABRICKS_MODEL_PROVIDER_SERVICE=<uc-provider-service-name>
GTM_JUDGE_MODEL=claude-sonnet-4-6
GTM_DRAFT_MODEL=claude-sonnet-4-6
GTM_CLASSIFIER_MODEL=claude-haiku-4-5
GTM_REPLY_DRAFT_MODEL=claude-haiku-4-5

With a provider service, model IDs are provider-side IDs rather than system.ai.* names. GTM Brain adds the Databricks-Model-Provider-Service header and confirms that the service supports the Anthropic protocol. Provider services backed by Anthropic and Amazon Bedrock are supported.

Bundle variables expose the same configuration:

databricks bundle deploy -t prod \
  --var="model_provider_service=<name>,judge_model=claude-sonnet-4-6,draft_model=claude-sonnet-4-6,classifier_model=claude-haiku-4-5,..."

OpenAI-compatible serving endpoint

This compatibility path is useful for a custom or legacy serving endpoint:

GTM_LLM_PROVIDER=databricks-serving-endpoint
GTM_LLM_BASE_URL=https://<host>/serving-endpoints/<endpoint>/invocations
GTM_LLM_API_KEY=<token>
GTM_JUDGE_MODEL=<served-model-name>

The implementation uses the AI SDK OpenAI-compatible provider. Prefer Gateway v2 for new Databricks deployments because it provides workspace-native auth, model-service discovery, and centralized usage/governance.

Direct Anthropic

GTM_LLM_PROVIDER=anthropic-direct
ANTHROPIC_API_KEY=<secret>
GTM_JUDGE_MODEL=claude-sonnet-4-6

This is useful outside Databricks or for isolated development. It bypasses Databricks Gateway usage tracking and UC provider governance, so it is not the recommended production route here.

AI chat versus GTM agents

The tenant chat and GTM judge/drafter/classifier share AI SDK foundations but serve different purposes. Chat can inspect the tenant's versioned strategy, automation status, and communication summary. Its only mutation is a bounded dry-run request for an already-active campaign version: the operator must approve the exact tool input in the chat, the server rechecks tenant membership and campaign state, and Temporal receives live: false. Chat cannot approve drafts, enable delivery, or send first-touch or reply email.

GTM production decisions still go through explicit model-role variables, telemetry, policy gates, and audited actions. Configure chat through the same server-side Databricks-authenticated provider factory if you want identical gateway policy and tracing; never place a workspace token in browser code.