Skip to content

adr-provider-agnostic-llm-facade

Decision

Adopt the Provider-Agnostic LLM Facade as a mandatory architectural pattern across all CAS agent workflows and orchestration modules.

Context

  • Source: governance-flow
  • Pattern: provider-agnostic-llm-facade
  • Promotion Recommendation: recommendation-provider-agnostic-llm-facade
  • Scope Classification: global
  • Scope Classification Artifact: pattern-scope-classification

Agentic systems often depend on closed model APIs or proprietary vendor SDKs. Switching providers or executing in offline CI/CD environments creates friction, vendor lock-in, and inconsistent error handling. The Provider-Agnostic LLM Facade encapsulates large language model invocations behind a cohesive interface (src/cas_runner/llm/base.py), isolating agent execution from vendor churn while providing symmetrical offline test doubles (MockLLMClient).

Scope

Applicability: Mandatory for all CAS components, agent steps, and workflow runners invoking large language models.

Applies across local development, offline CI/CD, and production multi-cloud deployments.

Consequences

Governance

Eliminates direct external SDK dependencies in domain and workflow logic. Enforces token budgeting, distributed tracing, and centralized review of model integrations.

Execution

Zero-overhead polymorphic dispatch via Python protocols. Dynamic routing across LiteLLM and hermetic mock test doubles for CI environments.

System Behaviour

Ensures deterministic fallback on API failures, normalized error handling, and centralized prompt/response caching.

Evidence Summary

Promotion Evidence

  • Recurring implementation across FlowRunner and AgenticMerger modules
  • Concrete AST symbols in src/cas_runner/llm/litellm_client.py (LiteLLMClient) and src/cas_runner/llm/mock_client.py (MockLLMClient)
  • Proven multi-provider test coverage across CI/CD environments
  • Governance model alignment with deterministic CI test doubles

Promotion Criteria

Significance, reusability, recurrence, evidence quality, governance maturity, and strategic alignment are fully met.

Governance Risks

Vendor abstraction leakage if vendor-specific parameters or proprietary prompt capabilities bypass the gateway contract.

Traceability

Pattern Proposal

provider-agnostic-llm-facade

Scope Classification

Artifact: pattern-scope-classification

Classification: global

Promotion Recommendation

recommendation-provider-agnostic-llm-facade

Promotion Decision

promotion-decision-provider-agnostic-llm-facade

Next Step

  • Maintain LiteLLMClient and MockLLMClient contracts as standard CAS LLM execution interfaces.
  • Institutionalize global pattern under knowledge/global/patterns/provider-agnostic-llm-facade.md.