Skip to content

Rule-Grounded Compliance Evaluation

Intent

Evaluate codebase deltas and repository state against codified deterministic rule specifications to produce machine-readable, auditable compliance findings rather than relying on subjective or non-deterministic model assessments.

Context

Autonomous and continuous architecture workflows require objective, machine-auditable verification of codebase deltas across delivery pipelines and pre-merge validation gates. Pure static linters lack architectural intent awareness, while unconstrained LLM assessments introduce hallucinations, non-determinism, and subjective drift across reviews.

Agentic Profile

Autonomous agents and CI automation tools execute compliance engines against discrete delta sets, receiving structured findings and diagnostics rather than open-ended text critique.

Forces

Balancing the need for high-velocity continuous architectural evaluation against the flakiness and subjectivity of unconstrained LLM reviews.

Drivers

  • Need for repeatable, machine-auditable verification of architectural standards in CI/CD pipelines
  • Requirement to eliminate human review velocity bottlenecks without sacrificing compliance rigor
  • Necessity of structured diagnostic outputs to guide automated or developer remediation

Constraints

  • Rule specifications must be deterministic and codified in structured domain schemas
  • Compliance verification must execute within continuous delivery gates with zero external model dependency

Tradeoffs

  • Upfront investment required to formalize and codify architectural policies into machine-executable rules
  • Maintenance overhead and risk of rule inflation or contradictory constraints as organizational rules expand

Solution

Isolate architectural compliance auditing into an explicit ComplianceEngine that evaluates diffs and state deltas against a catalog of codified, deterministic rules. The engine computes structured findings (ComplianceFinding) with explicit diagnostic reasons, severities, and metrics (ComplianceMetrics), providing deterministic pass/fail results across pipelines.

Operationalisation

Operationalised as an automated gating step in continuous integration workflows and pre-merge gates, validating changes against codified rule sets and populating structured compliance metrics.

Workflow Implications

  • Pre-merge gates and CI workflows (such as cas-build-validation and cas-continuous-architecture) execute rule evaluations against codebase deltas
  • Violations yield actionable diagnostic findings that halt pipeline progression or inform remediation steps
  • Requires periodic governance reviews to prune, consolidate, and deprecate redundant or conflicting rule definitions

Automation Opportunities

  • Automated aggregation of compliance findings into repository health dashboards
  • Zero-dependency CI rule execution removing external API latency and cost
  • Autonomous remediation suggestions generated directly from structured finding diagnostics

Implementation Signals

Implemented via ComplianceEngine in src/cas_runner/governance/compliance_engine.py and verified across continuous architecture pipelines.

Evidence

Empirically validated in production continuous architecture pipelines with high recurrence and strong governance alignment.

Signals

  • Validated in src/cas_runner/governance/compliance_engine.py
  • Active gating in cas-build-validation.yml and cas-continuous-architecture.yml
  • Consolidation of recurring proposals for schema-enforced compliance and health scoring

Confidence Basis

Verified implementation in core governance runner with production pipeline usage and strong alignment with state models.

Consequences

Establishes objective, reproducible verification of architectural invariants while introducing the operational need for ongoing rule lifecycle management.

Benefits

  • Deterministic, zero-LLM-cost verification of architectural compliance during CI runs
  • Eliminates hallucination and non-deterministic variance in architectural gatekeeping
  • Generates structured, machine-parsable finding metrics suitable for dashboard reporting and automated remediation

Risks

  • Rule inflation leading to maintenance overhead or mutually contradictory constraints
  • False positives or developer friction if rule definitions do not evolve alongside legitimate architectural changes

Governance Implications

  • Authoritative gating of pull requests and pattern promotions based on deterministic compliance scoring
  • Necessity of periodic governance reviews to audit, prune, and consolidate the active rule catalog

Reuse Notes

Applicable to all continuous integration pipelines, pre-merge gate validations, and repository governance verification flows across multi-repository workspaces.

Confidence

High confidence backed by active production implementation and comprehensive CI pipeline integration.

Level: high

Rationale

Validated in production runner code (ComplianceEngine), backed by active CI workflows, and supported by formal ADR acceptance.