SENTINEL SIGNAL

Sentinel Signal

Verify follow-up: alert truth and fallback readiness

Source: docs/mcp-verify-follow-up-alert-truth-implementation.md

All documents Docs Home Home

Document Content

Verify follow-up: alert truth and fallback readiness

Outcome

Implemented the follow-up consistency fixes for alert counting, publisher attribution, technical compatibility, and cold fallback readiness. The change does not alter score thresholds, production-readiness classification rules, database schemas, or public schema versions.

Changes

  • The server-page "High/critical-severity alerts" value now renders only

    • production_readiness.critical_alerts, which is derived from active_alerts. Remediation priorities no longer contribute to an alert count.

  • Remediation generation now has one fail-closed publisher-evidence gate. If

    • initialize received no response, or the registry URL returned non-MCP content, publisher-addressed check, score-category, and alert-derived rows are suppressed. Registry endpoint corrections and alias corrections remain.

  • Client profile scores remain criteria-based technical compatibility results.

    • The broader client-readiness verdict continues to apply policy, transport, request-association, and write-surface blockers separately. The page labels and explanatory copy now make this boundary explicit.

  • Fast report, policy, page, compact-compare, and trust-summary builders attach

    • fallback readiness and evidence fields before building snapshots. Cold metadata-only responses therefore carry the canonical metadata_only classification and reason; other partial responses carry evaluation_only instead of an all-null production_readiness_class.

Regression coverage

  • Alert-only count invariants, including a Tracepass-shaped low-alert/high-

    • remediation case.

  • No-response and non-MCP remediation suppression, including score-derived and

    • publisher-alert rows while registry corrections remain.

  • Technical 100.0 / compatible / 9 of 9 alongside a separately blocked

    • client-readiness verdict with a concrete policy blocker. The earlier missing step-up-auth 88.9 fail-closed regression remains in place.

  • First cold fallback responses for generic partial and metadata-only servers

    • across report, policy, compare, and trust-summary snapshot shapes.

Verification

  • PYTHONPATH=verify/src:verify/tests python3 -m pytest verify/tests -q:

    • 791 passed.

  • python3 -m pytest -m unit -q: 194 passed, 111 deselected.

Release verification

Deployed to IONOS on 2026-08-13 as version 1.0.624, commit f4a6849527964ec3154199651461c67ff2bdf577. The deployment restarted sentinel-signal.service, invalidating all in-memory route caches. The deploy script verified the version and commit on all six smoke routes, and /v1/build reported regression_checks_passing: true.

Cold and warmed responses were inspected for:

  • awesome-malinoto/tracepass-mcp-server
  • ryanatindago/playwright-mcp-example

Observed results:

  • Tracepass's first cold report was partial with a complete evaluation_only

    • readiness class. Its warmed report was safe_for_evaluation, contained one low card_omits_tool_schemas alert, and retained a high publisher fix_transport_compliance remediation. The rendered high/critical alert count was 0. Its technical profiles were 100.0 / compatible / 9 of 9 and 100.0 / compatible / 6 of 6, while both client-readiness verdicts remained separately blocked with concrete policy and transport reasons.

  • Playwright's cold and warm snapshots both carried the canonical

    • metadata_only code, label, and reason. Its warmed report contained one medium registry_endpoint_invalid alert and only four medium, registry-addressed corrections. It contained no publisher-addressed, score-derived, Critical, or High remediation rows. The rendered high/critical alert count was 0.

  • Cold-to-warm policy promotion was also verified. Tracepass moved from

    • evaluation_only to safe_for_evaluation; Playwright remained metadata_only. Every inspected readiness class had non-null code, label, and reason fields.