Skip to main content
Wardin has two caching layers. They are distinct mechanisms — don’t conflate them.

​
L1: Exact-match cache

A Redis-backed cache keyed on a deterministic hash of the request body (model + messages + params). On a hit, the cached response is returned immediately at $0 cost — the request never reaches the provider. Eligibility — a request is cache-eligible when all of these are true:
  • temperature is explicitly set to 0 (omitted temperature uses the provider’s non-deterministic default, so it is not cacheable)
  • No streaming ("stream": false or omitted)
  • No tools / function calling (an empty tools array is fine)
Cache hits are marked with the X-Wardin-Cache: HIT response header. Tenant scoping — cache keys are always prefixed with tenant_id. A cross-tenant cache hit is impossible by construction. Dashboard — the ROUTE stage shows the cache split (exact vs. semantic hits) with savings calculated at the model’s full input price; exact and semantic are never blended into one number.

​
L2: Semantic cache (pgvector)

A vector similarity cache backed by pgvector. When an incoming prompt has no exact match, its embedding is compared against stored embeddings. If similarity ≥ the configured threshold (default 0.92), the cached response is returned. Semantic hits are approximate — the response was generated for a similar but not identical prompt. The dashboard separates these from exact hits because they carry approximation risk. Semantic hits are marked with the X-Wardin-Cache: SEMANTIC response header. Threshold — configurable per tenant in Console → Cache. Raise it for more conservative matching; lower it for a higher hit rate with more approximation. Threshold changes are recorded to the EVIDENCE audit trail. Bypass — set "stream": true, include tools, or use a non-zero (or omitted) temperature to bypass both cache layers for that request.

​
Provider-native prompt caching (Anthropic)

This is not Wardin’s cache — it’s Anthropic’s server-side cache_control feature. Wardin passes cache_control blocks through unmodified and parses the resulting token types from the response separately:
Token typeCost multiplierWardin field
cache_creation_input_tokens1.25×cache_creation_cost_usd
cache_read_input_tokens0.1×cache_read_cost_usd
Regular input_tokens1.0×input_cost_usd
If these are blended, cost dashboards for agentic traffic (Claude Code, Cursor) will be significantly wrong. Wardin never blends them.

​
Cache metrics

Total savings = (exact_hit_tokens × model_input_price)
              + (semantic_hit_tokens × model_input_price)
              + (cache_read_tokens × (full_price - read_price))
The dashboard shows these three numbers separately so you know which savings carry zero risk (exact) vs approximation risk (semantic) vs provider-level savings (prompt cache).