Application and data architecture

Component topology

GTM Brain application compute runs in separate Cloudflare web and Temporal-worker Containers. Neon PostgreSQL is the authoritative application and Fabric Platform Host store, while Temporal Cloud retains durable workflow history and routing. The Cloudflare profile has no runtime dependency on a Databricks workspace, App, Lakebase endpoint, identity, secret, or URL. The stopped Databricks deployment and its provider adapters remain an optional, separately certified customer-managed target; they are not a fallback call path for Techfabric production. Static documentation remains independently deployed at https://gtm.fabric.pro.

A governed campaign pass

The workflow does not send merely because a model produced text. Policy evaluation, suppression, inbox ramp limits, idempotency, and human approval are first-class steps before delivery.

Fabric Platform mutation pipeline

GTM Brain uses @fabricorg/platform as its domain-governance runtime, not merely as a shared type package. The GTM module registers its ontology, actions, policies, state machine, adapters, and event contracts. The host executes the platform invariant:

Actor → ActionInvocation → PolicyEvaluation → StateMachine → Handler/Saga
      → AdapterInvocation → AssetEvent → Projection

Temporal provides durable orchestration around that pipeline; it does not replace it. A Monid result is fetched inside a Temporal activity, staged in Lakebase, and accepted only by invoking gtm.ingest_signal_v2 through the Fabric runtime. UI, agent, webhook, schedule, Apollo, and Monid inputs therefore converge on the same policy, audit, and event path.

Signal acquisition also has an explicit deadline boundary. New sweeps use a replay-patched 10-minute activity while granting provider adapters at most 105 seconds. Cancellation propagates into every Apollo, Instantly, and Monid request, and the activity checks its remaining window before each staged command. Provider timeout or cancellation therefore occurs before governed ingestion begins (or between complete commands), never as an unobserved provider continuation after Temporal has closed the activity.

The repository contract rejects authoritative store writes from API controllers, agent tools, scripts, Temporal workflows/activities, and chat tools. The only direct persistence exception is private transport/recovery staging that cannot alter a business projection and is consumed by an immutable, hash-checked action. Qualification decisions, delivery ledgers, reply sends, outbound sends, touches, record imports, and operational webhooks are registered actions; every external provider call is a declared Platform adapter.

The contract is semantic, not a text search. It resolves typed GtmStore and ProofAssetStore calls through computed access, destructuring, aliases, imported helpers, transitive call edges, and callbacks forwarded through dispatcher parameters. TypeScript overload declarations are mapped to their single executable implementation before mutation edges are built; missing or ambiguous overload resolution fails certification rather than dropping the call edge. External or injected dispatchers without an executable repository body conservatively treat directly supplied callback arguments as invoked, so collection, event, and rest-parameter APIs cannot hide a mutation helper. Exact allowlists cover only registered action implementations, Platform Host/adapters, and bounded private staging; stale exceptions fail the test. It also verifies adapter declaration/runtime bijection, default-deny external action exposure, caller-issued command identities, event phases, and the absence of principal proofs from new Temporal histories.

For any governance, recovery, authorization, migration, or provider-boundary change, certification requires the complete GTM suite both in memory and against PostgreSQL 17. A skipped PostgreSQL run is not release evidence. The database-backed run covers migrations, atomicity, idempotent consumption, terminal conflicts, tenant isolation, retention, bounded deletion, concurrency, and crash recovery. Pull requests expose these checks as the Governed mutation certification status. Protect main by requiring that exact status: it combines the semantic source scan, PostgreSQL 17 conformance, API admission/authorization, finite Harness-agent tests, and Fabric Experiments BDD. The Phase 3 BDD feature also proves that a retry resolves to one Platform action invocation and one AssetEvent, while a changed payload cannot reuse an admitted command identity. Run the principal gates with:

pnpm --filter @repo/gtm test
GTM_TEST_DATABASE_URL=postgresql://... pnpm --filter @repo/gtm test
pnpm --filter @repo/api test
pnpm --filter @repo/gtm-agents test
pnpm --filter @repo/gtm-worker test
pnpm type-check
pnpm lint
pnpm format:check
pnpm build
git diff --check

Durable command boundary

All domain-changing entry points use Temporal: governed campaign-operator requests, approval signals, manual outcomes, strategy and reply-definition changes, artifact/agent control-plane changes, inbound signal/reply webhooks, external-agent actions, and integration-secret changes. Credential plaintext is sealed before durable staging; Temporal receives only an opaque stage ID, and the canonical event contains no secret. API reads remain synchronous. Idempotent actions retry automatically; non-idempotent actions retain durable failure history but are never replayed blindly.

External-agent approval commits a deterministic provisioning dispatch in the same Lakebase transaction. The dispatcher starts the V2 provisioning workflow with a binding-derived workflow ID, making a lost start acknowledgement recoverable. The operator verifies the exact human approval, creates the registration suspended, stages one immutable tenant-encrypted credential generation, invokes the canonical registration and integration-secret actions, and records claim-ready status through gtm.complete_external_agent_provisioning. Claim atomically consumes the reveal marker and commits an activation dispatch; only that operator enables the registration. Revocation increments the request fence before the symmetric offboarding operator suspends the registration and revokes its exact credential generation. Only hashes and tenant/dispatch/workflow/run/staging/invocation IDs cross the Temporal boundary. The repository mutation contract explicitly limits these activities to bounded private staging; all externally meaningful transitions remain Platform actions.

Every mutation starts with a caller-issued command or request ID. The browser creates it once when the operator commits an intent; API, MCP, webhook, schedule, CLI, and agent callers supply an equivalent stable identity. The server does not generate a random fallback during dispatch. Same identity plus the same canonical payload returns the original invocation, while changed parameters conflict. Provider idempotency keys are derived from that governed identity.

Campaign execution begins with gtm.request_campaign_pass. Fabric Platform admits the operator request, then Temporal starts the exact V2 child pass using the immutable campaign version and validated execution manifest. Schedulers, UI starts, AI chat tools, and CLI starts converge on this operator; only replay-safe legacy history retains the former direct branch. Dry-run requests may be initiated by a registered agent. A live request must come through the admin-authenticated API and carry a short-lived, HMAC-signed campaign.operator capability bound to the tenant, natural-person actor, and exact command ID. The CLI cannot start live delivery.

The scheduler does not receive implicit authority. It first creates a completed, hash-bound gtm.request_campaign_pass invocation. The operator verifies that exact admission before launching the child pass. External agent admissions are immutable, registration-bound, single-consumption records and are revalidated against current grants and indirect campaign/resource ownership at the final Host seam. New API and gateway producers place only the opaque verified admission invocation ID in Temporal history; the HMAC proof, nonce, identity, and validity window remain outside workflow inputs. A replay-only activity branch continues to accept the original compact campaign command for histories recorded before admission moved ahead of orchestration. No current producer emits that legacy shape.

The external action catalog is default-deny. It is the intersection of the canonical Platform registry, an explicitly reviewed external-capability allowlist, and the authenticated registration's current grants. Adding a new action does not silently give external agents a new tool. The certified external action certification records a static reviewed fingerprint for every eligible action, including its normalized Platform schema, version, authority, resource scope, retry/idempotency classification, event phase, emitted events, and adapters. Runtime catalog construction and admission recompute the fingerprints. The generator produces a separate candidate and exact diff; updating that candidate does not certify it. Independent security/governance review is the default. A source-controlled single_owner_development_v1 exception currently permits its named owner to provide review evidence through October 31, 2026; runtime catalog, admission, and execution checks fail closed after that deadline. New, removed, or changed contracts are quarantined until the committed map is recertified for an exact clean SHA under the active mode. Admissions persist the exact per-action fingerprint separately from the catalog release digest; final execution revalidates that exact action binding. The out-of-process external-agent definition must match that reviewed external fingerprint and catalog release. Repository-owned finite Harness agents use the same admission record and final Host check, but bind the currently loaded governed action fingerprint to an action-scoped internal release digest. This keeps internal agent execution fail-closed on action drift without making internal actions externally discoverable or coupling an in-flight internal run to unrelated catalog changes.

External enrollment uses migration 50's tenant-composite invite, request, encrypted reveal, dispatch, operator-admission, and rate-limit tables. The public submit and claim boundaries validate raw secrets, reduce them to environment- and purpose-separated HMACs, and enter Platform as request-bound system actors. Human approval binds a narrowed authority envelope before the operator composes the canonical registration and integration-secret actions. The effective-access preview evaluates the same final authorization function using an ephemeral in-memory admission and does not write an invocation or create authority. Operator business-command IDs remain stable for Platform Host idempotency, while admission, actor, and private staging IDs are unique to the exact Temporal workflow run. Activities verify that run against Temporal's execution context. A reset therefore recovers the original governed invocation through a new one-use admission instead of weakening the run binding or creating a duplicate mutation. Operational event, invocation, approval, and run views apply the current campaign grant even when the registration originally created the record. Removing campaign scope therefore revokes both future execution and operational visibility for that campaign. Event and account-activity projections also fail closed when an event names a Platform invocation that cannot be loaded: the missing invocation is not treated as public or as proof of registration ownership. This rule applies equally to direct events, account activity, and synthetic imported-account activity.

Governed commercial experiment lifecycle

Phase 5 reuses Fabric Experiments' public lifecycle actions inside GTM Brain's existing Platform Host. There is one mutation pipeline and one durable invocation ledger—Fabric Experiments does not create a parallel controller or write path.

The tenant-admin endpoint POST /gtm/commercial-experiments/mutate accepts a stable commandId, one of the reviewed lifecycle or activation action IDs, and that action's schema-validated parameters. API admission binds tenantId to the organization and binds experiment owner, approver, or activator identity to the current operator. Temporal carries only staged, hash-checked command material. Lakebase migrations 44–48 store the tenant-scoped experiment aggregate, immutable lifecycle events, monotonic manifest version, activation binding, campaign treatment-consumption contract, immutable assignments, draft-consumption evidence, and actual-delivery exposure. GET /gtm/commercial-experiments returns the current tenant's plans, privacy-minimized lifecycle and activation evidence, exact campaign/artifact bindings, per-experiment assignment, draft-consumption, active-exposure, and retained-exposure counts, plus the tenant-wide attribution-fix count. It does not enumerate raw assignment subjects, touch IDs, delivery attempts, or tombstones.

Plan creation is stricter than the generic Experiments schema. Metadata must contain one exact gtm-commercial-outcome-experiment-v1 plan, both server-side variants must reproduce its immutable treatment bindings, browser JavaScript/CSS/DOM mutation payloads are prohibited, manual exposure is required, and each treatment must reference a currently certified artifact-generator version.

The lifecycle is activation-bound:

create → submit_for_review → approve
                                  │
             accepted source-controlled decision tuple
                                  ↓
       governed activation → start → internal account assignment
                                         │
                              server-resolved draft consumption
                                         │
                              governed approved delivery
                                         │
                       atomic attempt + touch + exposure
                                         │
                         human correction / governed retention

gtm.commercial_experiment_activation.v1 blocks every experiment.start until a later governed activation record binds the approved exact treatment tuple and gtm-commercial-experiment-consumption-v1 contract. The contract names one immutable active campaign version and two exact approved email-sequence revisions produced by the exact certified treatment versions/runs. The activation is created only through gtm.record_commercial_experiment_activation_v1 with the exact plan/contract binding hashes and immutable decision-record ID/hash. The tuple must already exist in the reviewed source-controlled activation registry, treatment certifications and artifact provenance are revalidated at activation, start, assignment, and draft resolution, and the production registry is currently empty.

After an activated experiment starts, the internal-only gtm.assign_commercial_experiment_treatment_v1 action uses Fabric Experiments' @fabricorg/experiments-node client with manual exposure. It persists one tenant-bound, no-crossover account assignment. Durable rows and audit events retain an opaque account-cluster hash, not the raw internal account ID. The campaign-pass activity resolves this assignment server-side and passes only its opaque ID to gtm.draft_outreach_v2. The draft action recalculates the expected assignment, selects the exact treatment artifact, and atomically persists the draft, consumption row, and GtmCommercialExperimentTreatmentConsumed event. Callers never supply a variant.

Draft consumption is not exposure. For a commercially bound draft, gtm.send_approved_outreach_v2 revalidates the running activation, current certification, immutable campaign/artifact provenance, assignment, and draft-consumption row before provider delivery. After provider acceptance, the adapter constructs the canonical Fabric Experiments exposure for the exact account-cluster assignment. Lakebase migration 47 commits the delivery-attempt sent transition, touch/cooldown row, and exposure in one transaction. Any missing or conflicting exposure rolls back the touch and leaves the provider-accepted attempt available only for governed reconciliation. GtmOutreachSent and GtmCommercialExperimentTreatmentExposed are after_adapters attestations and become visible only after that transaction succeeds.

The live checks authorize only a new provider call. Once the provider has accepted the stable request, a later experiment pause, artifact revocation, or certification change stops new sends but cannot suppress the factual delivery record. Governed recovery reconstructs the exact immutable activation, assignment, artifact, and consumption binding and uses the checkpointed provider acceptance time for the touch and exposure. Exact replay verifies the persisted touch and exposure; it never calls the provider again.

Attribution correction never edits the original outcome or exposure. gtm.correct_outcome_v2 accepts a replacement touch only for a natural-person manual attribution_fix, and verifies that it is the exact sent V2 delivery touch for the same company and tenant. It emits both the previous and effective touch IDs. The action is excluded from the external catalog.

Exposure evidence uses the audit_24m class under gtm-retention-policy-v1. Migration 48 stores the calendar-safe deadline and privacy-minimized tombstones. The daily Temporal dispatcher invokes gtm.enforce_commercial_experiment_exposure_retention_v1 through Platform Host with a stable tenant/day idempotency key. Each transaction processes at most 250 rows, recovers the same tombstones or exact empty result from an atomic run marker after interruption, rejects changed parameters, and prevents late delivery replay from resurrecting expired evidence.

All seven experiment lifecycle actions and all three commercial GTM actions are absent from the external-agent catalog. Hermes and other external runtimes cannot configure, activate, assign, or operate the experiment through their GTM account.

Approved V2 delivery atomically reserves tenant, campaign, account, and provider capacity. The attempt binds the exact contact ID, normalized recipient address, provider, and stable request identity. Provider acceptance is checkpointed, and the attempt becomes sent only when the required touch/cooldown row and, for commercial drafts, exact Fabric exposure commit in the same Lakebase transaction. Ambiguous outcomes remain capacity-consuming and are resolved through the governed reconciliation action; they are never blindly retried. Reconciliation repairs the same exact touch and exposure from the stored provider identity without sending twice.

The same conservative recovery rule applies to replies and approval notifications. A reply attempt left prepared after an adapter call is treated as potentially accepted and can be resolved only through the governed evidence-bound reconciliation action. Approval notifications persist one content-bound recipient checkpoint before compound completion, and unresolved provider effects are retained without a generic TTL until reconciliation makes them terminal.

Standalone provider effects such as governed chat persistence use the dedicated gtmProviderEffectWorkflowV2. It accepts only gtm.execute_provider_effect_v2, carries the opaque staged-command identity, and invokes the same Platform Host activity as other V2 work. The generic command workflow remains fail-closed for every internal-only action. Standalone AI requests use the matching gtmModelInferenceWorkflowV2, restricted to gtm.execute_model_inference_v2. After the API authenticates and authorizes the initiating person or agent, execution is delegated to the minimum system:provider-effect-runtime or system:gtm-model-runtime service principal. The invocation and completion event retain the initiating actor identity, preserving attribution without granting direct access to either internal action.

Tenant skill execution follows the same invariant. gtm.request_skill_run is the canonical admission action and binds the requesting principal, exact active skill version/hash, typed input hash, active knowledge hashes, risk/approval policy, and stable caller idempotency key. Raw input, knowledge, and output stay in tenant-isolated Lakebase staging; Temporal receives only opaque IDs and hashes. The compiled finite skill-runner has no tools, commands, or network access and can return only typed output plus inert proposals. A proposal becomes a business mutation only through a later, separately authorized Platform action invocation.

The registration, exact skill grant, runner identity, skill/knowledge state, and budgets are revalidated at execution time. Immutable staged output makes post-model retries deterministic, while bounded cleanup removes private skill inputs and outputs after 30 days. Compact audit events and run attestations retain the hashes and action references required for recovery without retaining private content. Credential-shaped skill input is rejected before invocation creation.

Agent package boundary

@repo/gtm-agent-contracts contains infrastructure-free schemas and ports. Both @repo/gtm and the root @repo/gtm-agents package depend on those contracts; GTM core does not depend on Fabric Harness. The @repo/gtm-worker composition root imports both and supplies tenant-scoped ports.

Built-in agent installation is a governed control-plane operation. Stable definitions are installed once; every prompt/tool/model/budget configuration receives a content-derived immutable version ID. Tenant registrations point to one version and can be disabled, suspended, campaign-scoped, or rebound to a newly published version. The Temporal activity resolves that version before invoking Harness, so the model and budget displayed in the UI are the values actually enforced at runtime.

OpenKnowledge is intentionally not an operational authority. Databricks AI Search provides the current tenant-isolated semantic retrieval adapter; writes still return through GTM's revision/hash actions. The Delta chunk table and continuous index are rebuildable projections of active workspace Knowledge. Every candidate is tenant-filtered in Databricks and then revalidated against the exact canonical Lakebase revision before disclosure. Backfill and repair use the internal-only gtm.reindex_knowledge Platform action; maintenance scripts never write the Delta source directly. Because OpenKnowledge is GPL-3.0-or-later and selects projects by working directory, any future editing integration still requires legal approval plus one isolated process/container and fixed root per tenant (or a GTM-owned proxy that never accepts arbitrary cwd). Its session activity panel cannot replace the durable GTM agent/run/action ledger.

Operational and analytical planes

Lakebase is the low-latency operational system of record. It contains organizations, auth sessions, integrations, accounts, signals, drafts, approvals, event envelopes, and invocation audit records. The deployment migration identity owns schema changes for both GTM and the Fabric Platform Host ledger. Databricks App identities receive runtime DML only. Bundle reconciliation can replace the OAuth role, so every deployment reapplies the reviewed runtime grant before App promotion. In preprovisioned mode, one startup preflight validates the GTM migration ledger plus all required fabric_platform tables and privileges before either the web server or Temporal worker starts; it never runs schema DDL during application startup.

Unity Catalog is the analytical and governance plane. The hourly serverless job reads the Lakebase UC catalog and idempotently merges operational event envelopes into gtm_brain_analytics.gtm.events. Funnel tables, evaluation tables, App telemetry, and MLflow traces live beside the projection without placing analytical load on the application database. The knowledge_chunks Delta table is a separate content-bearing projection with change data feed enabled; the App receives SELECT on the exact AI Search index plus SELECT and MODIFY on that source table, and writes it only through the declared Knowledge action adapters. Agents never receive that Databricks identity.

Failure boundaries

• A failed model call records an error and does not crash the whole pass.
• A failed deployment does not replace the last successful App snapshot.
• Temporal persists workflow state outside App compute, so an App restart does not lose orchestration.
• Stable command keys make webhook and external-agent redelivery converge on one workflow execution.
• Lakebase persists operational state and audit history across worker restarts.
• The projection uses an idempotent Delta MERGE; replay does not duplicate events.
• Delivery requires an approved draft and vendor credentials scoped to the active organization.