Generalize the Codex Responses-to-Chat tool media mechanism from 6c9d444c
to the remaining protocol bridges, so image-bearing tool results are never
tokenized as base64 text on any conversion path:
- Claude-to-Chat: tool_result images become native image_url parts,
batched into one synthetic user message after the tool message batch.
- Claude-to-Responses: JSON-string, MCP, and nested variants are restored
as native input_image parts inside function_call_output.
- Codex/GrokBuild-to-Anthropic: non-standard tool images are restored as
Anthropic image blocks (case-insensitive data:/http prefix parsing).
- Claude-to-Gemini: Gemini 3 uses multimodal functionResponse.parts;
older models get inlineData parts in the same user turn. The new
InlineImagesOnly scope keeps remote URLs and malformed data URLs in the
legacy text form instead of emitting fileData the API would reject.
Shared changes in tool_media:
- Centralize the plan/queue/flush helpers used by both Chat bridges.
- strip_and_clamp_media_from_tool_value clamps residual base64 inside
parsed JSON strings at any nesting depth before re-serialization.
- Emitted Chat parts no longer carry cache_control or
prompt_cache_breakpoint, preserving the strip-all-cache_control
contract that strict upstreams (GLM/Qwen) depend on.
- Media detection now requires full convertibility, keeping detection
and extraction symmetric by construction.
The media sanitizer detects and strips the new shapes symmetrically
(Chat string tool results, Anthropic string tool results, Gemini
inlineData/fileData and functionResponse.parts); the legacy typed-block
replacement runs first so replacement stays a superset of detection and
cache_control survives on replaced Anthropic blocks.
No-media tool outputs keep byte-identical legacy representations on all
bridges to protect prompt-cache prefixes.
The Responses->Chat conversion serialized image-bearing *_output items
into role:"tool" text via canonical_json_string, so view_image results
were tokenized as base64 text (~9000x inflation). Codex replays full
history every turn, so sessions hit context-limit 400s and wedged
(#4465, #5663).
- add proxy/tool_media: shared detection/strip/clamp walker for tool
output media (typed input_image / image_url / input_file /
input_audio, Anthropic source and MCP data+mimeType image shapes,
untyped data: image_url, whole-string bare data URLs)
- transform_codex_chat: replace media blocks in place with marker text
so tool content stays a plain string, and flush the extracted media
as one synthetic role:"user" message after each consecutive tool
batch; media-free traffic stays byte-identical to keep prompt-cache
prefixes stable
- media_sanitizer: detect and strip tool-output media symmetrically
(including JSON-string outputs) so reactive image stripping can heal
upstream modality rejections
- forwarder: regression tests pinning the reactive trigger and the
context-limit-400 non-trigger
E2E against Kimi K3 through the proxy: the replayed turn stays ~12k
input tokens with 99% cache hit, versus ~85k+ of base64 text per
replay before.
Jason