fix(proxy): improve cache hit rate for Codex/Responses requests

prompt_cache_key was falling back to provider.id when the client did not
supply a session, which collapsed every conversation onto a single key
and defeated upstream prefix caching. Only emit the key when a real
client-provided session/thread identity is available; otherwise let the
upstream use its default matching behaviour.

Additional fixes that affect cache stability:
- Canonicalise (sort) JSON keys in outgoing request bodies and in
  tool_call arguments / tool_result content so semantically identical
  requests produce identical byte sequences for upstream prefix caches.
- Exempt JSON Schema property maps (properties, patternProperties,
  definitions, \$defs) from the underscore-prefix filter so user-defined
  schema keys like _id and _meta survive.
- Add a [CacheTrace] debug log with stable hashes for instructions,
  tools, input and include to help diagnose cache misses.
- Thread session_id into the usage logger for request correlation.
This commit is contained in:
Jason
2026-05-11 08:41:32 +08:00
parent aec055a1d1
commit 00a789e7a3
9 changed files with 392 additions and 31 deletions
+2 -2
View File
@@ -309,8 +309,8 @@ pub struct ProviderMeta {
pub is_full_url: Option<bool>,
/// Prompt cache key for OpenAI Responses-compatible endpoints.
/// When set, injected into converted Responses requests to improve cache hit rate.
/// If not set, Codex OAuth uses the current session ID; other Claude -> Responses
/// conversions fall back to provider ID.
/// If not set, Claude -> Responses conversions use a client-provided session/thread
/// identity when available; generated session IDs are not sent upstream.
#[serde(rename = "promptCacheKey", skip_serializing_if = "Option::is_none")]
pub prompt_cache_key: Option<String>,
/// Codex OAuth FAST mode: inject `service_tier = "priority"` for ChatGPT Codex requests.