Pattern Proposal: Dual-Tier LLM Gateway Abstraction
Summary
Decouples agent prompt invocation from specific providers through a LiteLLM adapter backed by a mock client for zero-cost testing.
Candidate Pattern
- ID:
dual-tier-llm-gateway-abstraction - Classification: intelligence
- Maturity: established
- Type: implemented
Observed Signals
No specific signals listed.
Evidence
Supporting Evidence
- src/cas_runner/llm/litellm_client.py (LiteLLMClient) - Production LLM client utilizing litellm router.
- src/cas_runner/llm/mock_client.py (MockLLMClient) - Deterministic test double for CI environments.
Scope Assessment
- Evidence Strength: high
- Confidence: high
- Applies to: ideas-to-life-cas
Rationale
- Intent: Enable transparent switching between real LLM providers and deterministic offline mocks without modifying agent logic.
- Context: Invoked whenever FlowRunner or AgenticMerger dispatches prompt tasks.
- Force: Continuous CI workflows cannot rely on live LLM API keys or incur unpredictable latency and cost.
- Force: Hardcoded provider SDKs make model upgrades and vendor diversification costly.
Traceability
Artifact source: proposed-patterns