Pattern Proposal: Provider-Agnostic LLM Gateway

Summary

Encapsulates LLM inference behind a unified client interface with mock fallbacks for deterministic evaluation and provider flexibility.

Candidate Pattern

• ID: provider-agnostic-llm-gateway
• Classification: intelligence
• Maturity: established
• Type: implemented

Observed Signals

No specific signals listed.

Evidence

Supporting Evidence

• src/cas_runner/llm/litellm_client.py (LiteLLMClient) - Production LLM client wrapping litellm.
• src/cas_runner/llm/mock_client.py (MockLLMClient) - Mock client providing deterministic responses for tests.

Scope Assessment

• Evidence Strength: high
• Confidence: high
• Applies to: ideas-to-life-cas

Rationale

• Intent: Shield agent logic from vendor-specific LLM APIs while facilitating offline mock testing and runtime provider switching.
• Context: Agent runner requiring multi-model support and repeatable continuous integration tests without live API dependencies.
• Force: Direct vendor SDK coupling locks the system into single providers and complicates rate-limit handling
• Force: Live API calls in CI/CD pipelines introduce flakiness, cost, and non-determinism

Traceability

Artifact source: proposed-patterns