Operations

Production rollout evidence — July 29, 2026

Certified release, governed recovery, projection parity, and non-delivery canary evidence for release d1998d70.

This record covers the production rollout of commit d1998d70e5901ec82b44f237d9ff9b0b4c474fe9. It records operational evidence; it does not authorize live delivery or another prospect-visible action.

Released artifacts

ArtifactExact identityResult
Git released1998d70e5901ec82b44f237d9ff9b0b4c474fe9Certified protected checks passed
Databricks App deployment01f18b199f11101f88b5a1d0e153a249SUCCEEDED, App RUNNING, compute ACTIVE
Temporal Worker Build IDgtm-prod-d1998d70-20260729T064739ZRegistered and current for gtm-brain-prod
Lakebase schema ledgermigration 49, 49 rowsExact expected ceiling; no release DDL required

The release came from a clean detached worktree at the exact merge commit. Frozen-lockfile installation and bundle validation passed before the bundle was deployed. Startup then reported a valid Prisma schema, a ready Next.js process, updated schedules, and RUNNING workers for both gtm-brain and gtm-brain-v2.

Authenticated /api/health returned 200 OK. The Databricks doctor passed all ten workspace, gateway, model, warehouse, analytics, MLflow, and TLS Lakebase checks. Temporal routing verification confirmed the exact Build ID after promotion.

Governed recovery evidence

Two provider effects were parked by failures that occurred before their provider callbacks:

  1. An Apollo integration diagnostic had been rejected by the previous strict action schema before adapter execution.
  2. An internal approval notification recorded AdapterInvocationStarted, then stopped when the old projection implementation rejected a sparse canonical sequence. The Host throws while appending that event, before entering adapter.execute.

Each terminal action/staging record and provider-effect binding was inspected before recovery. Neither effect had a provider request ID. Each was recovered once through gtm.reconcile_provider_effect_v2 with a new, deterministic, operator-issued command identity that described the prior failure. The commands did not reuse a terminal action identity and did not call a provider directly.

The resulting Platform invocations were:

  • act_01KYPARPN45Z4102QK4SCRXAWK for the diagnostic;
  • act_01KYPB92GKHH3Q7JHMEBK6NWWR for the approval notification.

Both completed with one GtmProviderEffectCompleted event. The approval notification's recipient-bound mail checkpoint completed with one stable provider request identity. After recovery, all eight production provider effects were terminal: five signal acquisitions, one integration diagnostic, one console notification, and one mail notification. No raw provider response, credential, token, or recipient value was added to canonical action evidence.

Projection and audit parity

After recovery, the Lakebase domain-mirror checkpoint and canonical Host ledger both ended at sequence 1325. The canonical ledger contained 1,324 events because sequence 1314 is intentionally absent; the sparse sequence is monotonic and must not be synthesized.

The production Delta projection job completed successfully after the final recovery:

CheckSourceDeltaMissing from DeltaRetained historical
Privacy-safe events1,5671,602035
Typed GTM lineage79579904

The retained rows predate the current Lakebase source-ledger cutover and were recorded on July 13 or July 16. The Delta projection is insert-only, so it preserves that historical audit evidence. No current source event or lineage row was missing, and both event projections ended at canonical sequence 1325.

Production canaries

  • Invalid agent credentials received the MCP JSON-RPC authorization error and no registration-filtered catalog.
  • Invalid webhook and internal-notification secrets returned 401.
  • A correctly authenticated webhook request for a deliberately nonexistent tenant returned 404 before parsing or mutation. That tenant created zero Platform invocations and zero events.
  • The internal approval-notification recovery completed through its declared Platform adapter and durable recipient checkpoint.
  • The existing campaign workflow remained RUNNING on gtm-brain-v2, live=false, and awaiting-approvals: shortlist 35, one pending draft, one pending approval, and zero resolved decisions.
  • Production retained zero delivery attempts and zero non-dry-run touches from the rollout window.
  • GTM_ENABLE_V2_LIVE_DELIVERY remained 0.

The canary deliberately did not approve the pending draft, create a prospect delivery, or enable autonomous prospect-visible actions. Those require a separate, explicit release decision after this evidence is reviewed.

On this page