Verify build-scoped route cache

Date: 2026-08-04 Updated: 2026-08-04 (Round 3 — TASK-08b, root-path cache fix)

Finding

Production GET responses are served directly by Caddy and Uvicorn. They carry Cache-Control, CDN-Cache-Control, and Surrogate-Control values of no-store, with no CDN Age, X-Cache, or provider cache headers. The stale HTML layer was therefore Verify's application route cache.

curl -I is not a valid cache diagnostic for these routes because Verify does not implement HEAD and returns 405. Use a GET while discarding the body:

curl -sD - -o /dev/null https://verify.sentinelsignal.io/ \
  | grep -iE '^(age|cache-control|cdn-cache-control|etag|x-cache|cf-cache-status|server|surrogate-control|x-mcp-verify)'

Implementation

Every application route-cache key is now scoped by:

1. The immutable deployed build SHA.
2. The request path.
3. A digest of the complete query string.
4. The route's existing data/snapshot key.

The effective shape is build SHA + path + query digest + route key. A new deployment therefore cannot read entries created by an older build, even when the public semantic version is accidentally unchanged.

Production should set MCP_VERIFY_BUILD_SHA to the deployed git commit. As a fail-safe, the Verify image creates /app/.build-sha from its packaged source, migration, and metadata contents during the Docker build. Local development falls back to the package/site version.

Stale-while-refresh responses are also bounded by MCP_VERIFY_ROUTE_CACHE_MAX_STALE_SECONDS, which defaults to 60 seconds beyond a route's normal TTL. Entries older than that are discarded instead of being served indefinitely.

All responses now expose X-MCP-Verify-Build, and /version plus /healthz include build_sha. API ETags also use the build SHA.

Root-path staleness recurrence (TASK-08b, Round 3)

The bare / path served a build up to nine revisions behind ?query variants across three consecutive deploys, resolved once by a manual cache purge and then re-frozen on the next deploy. Root cause was not a CDN (none is in front of Verify — confirmed again; DNS cutover to IONOS-only, 66.179.248.190, has been complete since 2026-07-31) and not the route-cache key logic itself (/ and /?query use the same build_scoped_route_cache_key() scoping). It was the deploy procedure: MCP_VERIFY_BUILD_SHA was a human-maintained environment variable, updated by a manual SSH + sed step that (a) silently no-oped if the key line was ever missing, and (b) had to run in a specific order relative to the container restart, with nothing enforcing either. Whenever that step was skipped or misordered, the env var stayed pinned to a previous release's SHA, and resolve_verify_build_sha()'s deterministic content-hash fallback (/app/.build-sha, baked at Docker build time) never engaged, because that fallback only fires when the env var is unset — not when it's stale.

Separately, and independent of the above, the startup route-cache prewarm pass (_prewarm_default_route_caches in verify/src/mcp_verify/main.py) wrote directly into the cache dictionaries using the pre-build-scoping key format, bypassing build_scoped_route_cache_key() entirely. Every prewarm write since the build-scoping change was therefore orphaned (an unreachable key next to the one real requests look up) — not the cause of the staleness, since a freshly started process's cache is empty regardless of key format, but wasted prewarm work and a red herring when debugging from cold. Fixed in the same pass as the cache-key scoping was fixed for sitemap.xml, /backfill, /, and /v1/servers.

Deploy procedure

The manual SSH/sed runbook has been replaced with deploy/ionos/deploy.sh, run on the IONOS host from an up-to-date checkout:

ssh root@66.179.248.190 "cd /opt/sentinel-signal && git pull && ./deploy/ionos/deploy.sh"

deploy.sh:

1. Reads git rev-parse HEAD and VERSION from the checkout that's about to be deployed.
2. Writes MCP_VERIFY_BUILD_SHA and MCP_VERIFY_SITE_VERSION into /etc/sentinel-signal/prod.env, idempotently (set-or-append — the old silent-no-op-on-missing-key failure mode is gone).
3. Restarts sentinel-signal.service.
4. Waits for verify-web to report healthy.
5. Runs scripts/verify_public_route_versions.sh with VERIFY_EXPECTED_BUILD_SHA/VERIFY_EXPECTED_VERSION set to the commit just deployed, and exits non-zero if any route disagrees — the deploy is not considered done until every smoke-checked route (/, /pricing, /trust-index, a server-detail page, /methodology, /status) reports the same build.

There is no separate manual smoke-check step anymore; it's step 5 above. If you need to run it by hand against a already-deployed release, make verify-route-version-smoke still works standalone.