9 Commits

Author	SHA1	Message	Date
Keith Yu	8217bfff50	feat: add Bedrock request optimizer (PRE-SEND thinking + cache injection) (#1301) ... * feat: add Bedrock request optimizer (PRE-SEND thinking + cache injection) Add a PRE-SEND request optimizer that enhances Bedrock API requests before forwarding, complementing the existing POST-ERROR rectifier system. New modules: - thinking_optimizer: 3-path model detection (adaptive/legacy/skip) - Opus 4.6/Sonnet 4.6: adaptive thinking + effort max + 1M context beta - Legacy models: inject extended thinking with max budget - Haiku: skip (no modification) - cache_injector: auto-inject cache_control breakpoints (max 4) - Injects at tools/system/assistant message positions - TTL upgrade for existing breakpoints (5m → 1h) Gate: only activates for Bedrock providers (CLAUDE_CODE_USE_BEDROCK=1) Config: stored in SQLite settings table, default OFF, user opt-in UI: new Optimizer section in RectifierConfigPanel with 3 toggles + TTL 18 unit tests covering all paths. Verified against live Bedrock API. * chore: remove docs/plans directory * fix: address code review findings for Bedrock request optimizer P0 fixes: - Replace hardcoded Chinese with i18n t() calls in optimizer panel, add translation keys to zh/en/ja locale files - Fix u64 underflow: max_tokens - 1 → max_tokens.saturating_sub(1) - Move optimizer from before retry loop to per-provider with body cloning, preventing Bedrock fields leaking to non-Bedrock providers P1 fixes: - Replace .map() side-effect pattern with idiomatic if-let (clippy) - Fix module alphabetical ordering in mod.rs - Add cache_ttl whitelist validation in set_optimizer_config - Remove #[allow(unused_assignments)] and dead budget decrement --------- Co-authored-by: Keith (via OpenClaw) <keithyt06@users.noreply.github.com> Co-authored-by: Jason <farion1231@gmail.com>	2026-03-07 18:57:21 +08:00
Dex Miller	f3343992f2	feat(proxy): add thinking signature rectifier for Claude API (#595) ... * feat(proxy): add thinking signature rectifier for Claude API Add automatic request rectification when Anthropic API returns signature validation errors. This improves compatibility when switching between different Claude providers or when historical messages contain incompatible thinking block signatures. - Add thinking_rectifier.rs module with trigger detection and rectification - Integrate rectifier into forwarder error handling flow - Remove thinking/redacted_thinking blocks and signature fields on retry - Delete top-level thinking field when assistant message lacks thinking prefix * fix(proxy): complete rectifier retry path with failover switch and chain continuation - Add failover switch trigger on rectifier retry success when provider differs from start - Replace direct error return with error categorization on rectifier retry failure - Continue failover chain for retryable errors instead of terminating early * feat(proxy): add rectifier config with master switch - Add RectifierConfig struct with enabled and requestThinkingSignature fields - Update should_rectify_thinking_signature to check master switch first - Add tests for master switch functionality * feat(db): add rectifier config storage in settings table Store rectifier config as JSON in single key for extensibility * feat(commands): add get/set rectifier config commands * feat(ui): add rectifier config panel in advanced settings - Add RectifierConfigPanel component with master switch and thinking signature toggle - Add API wrapper for rectifier config - Add i18n translations for zh/en/ja * feat(proxy): integrate rectifier config into request forwarding - Load rectifier config from database in RequestContext - Pass config to RequestForwarder for runtime checking - Use should_rectify_thinking_signature with config parameter * test(proxy): add nested JSON error detection test for thinking rectifier * fix(proxy): resolve HalfOpen permit leak and RectifierConfig default values - Fix RectifierConfig::default() to return enabled=true (was false due to derive) - Add release_permit_neutral() for releasing permits without affecting health stats - Fix 3 permit leak points in rectifier retry branches - Add unit tests for default values and permit release * style(ui): format ProviderCard style attribute * fix(rectifier): add detection for signature field required error Add support for detecting "signature: Field required" error pattern in the thinking signature rectifier. This enables automatic request rectification when upstream API returns this specific validation error.	2026-01-14 00:12:13 +08:00
Dex Miller	6dd809701b	Refactor/simplify proxy logs (#585) ... * refactor(proxy): simplify logging for better readability - Delete 17 verbose debug logs from handlers, streaming, and response_processor - Convert excessive INFO logs to DEBUG level for internal processing details - Add 2 critical INFO logs in forwarder.rs for failover scenarios: - Log when switching to next provider after failure - Log when all providers have been exhausted - Fix clippy uninlined_format_args warning This reduces log noise while maintaining visibility into key user-facing decisions. * fix: replace unsafe unwrap() calls with proper error handling - database/dao/mcp.rs: Use map_err for serde_json serialization - database/dao/providers.rs: Use map_err for settings_config and meta serialization - commands/misc.rs: Use expect() for compile-time regex pattern - services/prompt.rs: Use unwrap_or_default() for SystemTime - deeplink/provider.rs: Replace unwrap() with is_none_or pattern for Option checks Reduces potential panic points from 26 to 1 (static regex init, safe). * refactor(proxy): simplify verbose logging output - Remove response JSON full output logging in response_processor - Remove per-request INFO logs in provider_router (failover status, provider selection) - Change model mapping log from INFO to DEBUG - Change usage logging failure from INFO to WARN - Remove redundant debug logs for circuit breaker operations Reduces log noise significantly while preserving important warnings and errors. * feat(proxy): add structured log codes for i18n support Add error code system to proxy module logs for multi-language support: - CB-001~006: Circuit breaker state transitions and triggers - SRV-001~004: Proxy server lifecycle events - FWD-001~002: Request forwarding and failover - FO-001~005: Failover switch operations - USG-001~002: Usage logging errors Log format: [CODE] Chinese message Frontend/log tools can map codes to any language. New file: src/proxy/log_codes.rs - centralized code definitions * chore: bump version to 3.9.1 * style: format code with prettier and rustfmt * fix(ui): allow number inputs to be fully cleared before saving - Convert numeric state to string type for controlled inputs - Use isNaN() check instead of || fallback to allow 0 values - Apply fix to ProxyPanel, CircuitBreakerConfigPanel, AutoFailoverConfigPanel, and ModelTestConfigPanel * feat(pricing): support @ separator in model name matching - Refactor model name cleaning into chained method calls - Add @ to - replacement (e.g., gpt-5.2-codex@low → gpt-5.2-codex-low) - Add test case for @ separator matching * fix(proxy): improve validation and error handling in proxy config panels - Add StopTimeout/StopFailed error types for proper stop() error reporting - Replace silent clamp with validation-and-block in config panels - Add listenAddress format validation in ProxyPanel - Use log_codes constants instead of hardcoded strings - Use once_cell::Lazy for regex precompilation * fix(proxy): harden error handling and input validation - Handle RwLock poisoning in settings.rs with unwrap_or_else - Add fallback for dirs::home_dir() in config modules - Normalize localhost to 127.0.0.1 in ProxyPanel - Format IPv6 addresses with brackets for valid URLs - Strict port validation with pure digit regex - Treat NaN as validation failure in config panels - Log warning on cost_multiplier parse failure - Align timeoutSeconds range to [0, 300] across all panels	2026-01-11 20:50:54 +08:00
Dex Miller	5376ea042b	Feat/usage improvements (#508) ... * i18n: update cache terminology across all languages - Change 'Cache Read' to 'Cache Hit' in all languages - Change 'Cache Write' to 'Cache Creation' in all languages - Update zh: 缓存读取 → 缓存命中, 缓存写入 → 缓存创建 - Update en: Cache Read → Cache Hit, Cache Write → Cache Creation - Update ja: キャッシュ読取 → キャッシュヒット, キャッシュ書込 → キャッシュ作成 Affected keys: cacheReadTokens, cacheCreationTokens, cacheReadCost, cacheWriteCost, cacheRead, cacheWrite * feat(usage): add cache metrics to trend chart - Add cache creation tokens visualization (orange line) - Add cache hit tokens visualization (purple line) - Add gradient definitions for new cache metrics - Include cache data in hourly aggregation - Display cache metrics alongside input/output tokens This provides better visibility into cache usage patterns over time. * fix(usage): fix timezone handling in datetime picker - Add timestampToLocalDatetime() to convert Unix timestamp to local datetime - Add localDatetimeToTimestamp() with validation for incomplete input - Fix issue where typing hours/minutes would jump to previous day - Validate datetime format completeness before conversion - Use local timezone instead of UTC for datetime-local input This resolves the issue where users couldn't fine-tune time selection and the input would jump unexpectedly when editing hours or minutes. * feat(usage): add auto-refresh for usage statistics - Add 30-second auto-refresh interval for all usage queries - Disable background refresh to save resources - Apply to: summary, trends, provider stats, model stats, request logs - Queries automatically update when tab is active - Pause refresh when user switches to another tab This keeps usage data fresh without manual refresh. * fix(proxy): improve usage logging and cache token parsing - Log requests even when usage parsing fails (with default values) - Add detailed debug logging for usage metrics - Support cache_read_input_tokens field in Codex responses - Fallback to input_tokens_details.cached_tokens if needed - Add test case for cached_tokens in input_tokens_details - Ensure all requests are tracked in database for analytics This fixes missing request logs when API responses lack usage data and improves cache token detection across different response formats. * style(rust): use inline format args in format! macros - Replace format!("...", var) with format!("...{var}") - Update universal provider ID formatting - Update error message formatting - Update config.toml generation in Codex provider Fixes clippy::uninlined_format_args warnings. * feat(proxy): enhance provider router logging - Add debug logs for failover queue provider count - Log circuit breaker state for each provider check - Add logs for missing current provider scenarios - Log when no current provider is configured - Use inline format args for better readability This improves debugging of provider selection and failover behavior. * feat(database): update model pricing data - Update Claude models to full version format (e.g. claude-opus-4-5-20251101) - Add GPT-5.2 series model pricing (10 models) - Add GPT-5.1 series model pricing (10 models) - Add GPT-5 series model pricing (12 models) - Add Gemini 3 series model pricing (2 models) - Update Gemini 2.5 series model ID format (use dot separator) - Unify display names by removing thinking level suffixes * fix(usage): correct Gemini output token calculation Fix Gemini API output token parsing to use totalTokenCount - promptTokenCount instead of candidatesTokenCount alone. This ensures thoughtsTokenCount is included in output statistics. - Update from_gemini_response to calculate output from total - input - Update from_gemini_stream_chunks with same logic for consistency - Fix from_codex_stream_events to use adjusted token calculation - Add test case for responses with thoughtsTokenCount - Update existing tests to match new calculation logic * fix(usage): correct cache token billing and add Codex format auto-detection - Avoid double-billing cache tokens by subtracting from input before calculation - Add smart Codex parser that auto-detects OpenAI vs Codex API format - Extract model name from Codex responses for accurate tracking * fix(proxy): improve takeover detection with live config check - Add live config takeover detection for hot-switch decision - Rebuild takeover when backup is missing or placeholder remains - Make detect_takeover_in_live_config_for_app public - Fix is_takeover_active to use actual takeover status * refactor(usage): simplify model pricing lookup by removing suffix fallback Replace complex suffix-stripping fallback with direct prefix/suffix cleanup. Model IDs are now cleaned by removing vendor prefix (before /) and colon suffix (after :), then matched exactly against pricing table. * feat(database): add Chinese AI model pricing data Add pricing for domestic AI models (CNY/1M tokens): - Doubao-Seed-Code (ByteDance) - DeepSeek V3/V3.1/V3.2 - Kimi K2/K2-Thinking/K2-Turbo (Moonshot) - MiniMax M2/M2.1/M2.1-Lightning - GLM-4.6/4.7 (Zhipu) - Mimo V2 Flash (Xiaomi) Also fix test case to use correct model ID and remove invalid currency column. * refactor(proxy): improve header forwarding with blacklist approach Change from whitelist to blacklist mode for request header forwarding. Only skip headers that will be overridden (auth, host, content-length). This preserves client's original headers and improves compatibility. * fix(proxy): bypass timeout and retry configs when failover is disabled When auto_failover_enabled is false, timeout and retry configurations should not affect normal request flow. This change ensures: - create_forwarder: passes 0 for all timeout/retry params when failover is disabled, effectively bypassing these checks - streaming_timeout_config: returns 0 for both first_byte_timeout and idle_timeout when failover is disabled This prevents unnecessary timeout errors and retry attempts when users have explicitly disabled the failover feature. * fix(proxy): handle zero value input in failover config fields * refactor(proxy): remove retry logic and add enabled check for failover * refactor(proxy): distinguish circuit-open from no-provider errors * Align usage stats to sliding windows * feat(proxy): add body and header filtering for upstream requests * feat(proxy): enable transparent passthrough for headers - Passthrough anthropic-beta header as-is from client - Passthrough anthropic-version header from client - Passthrough client IP headers (x-forwarded-for, x-real-ip) by default - Filter private params (underscore-prefixed fields) from request body - No database changes required * feat(proxy): extract session ID from client requests for logging - Add SessionIdExtractor to parse session ID from Claude/Codex requests - Support extraction from metadata.user_id, headers, previous_response_id - Pass session_id through RequestContext to usage logger - Enable request correlation by session in proxy_request_logs	2025-12-31 22:57:00 +08:00
YoVinchen	e6f18ba801	Feat/usage model extraction (#455) ... * feat(proxy): extract model name from API response for accurate usage tracking - Add model field extraction in TokenUsage parsing for Claude, OpenAI, and Codex - Prioritize response model over request model in usage logging - Update model extractors to use parsed usage.model first - Add tests for model extraction in stream and non-stream responses * feat(proxy): implement streaming timeout control with validation - Add first byte timeout (0 or 1-180s) for streaming requests - Add idle timeout (0 or 60-600s) for streaming data gaps - Add non-streaming timeout (0 or 60-1800s) for total request - Implement timeout logic in response processor - Add 1800s global timeout fallback when disabled - Add database schema migration for timeout fields - Add i18n translations for timeout settings * feat(proxy): add model mapping module for provider-based model substitution - Add model_mapper.rs with ModelMapping struct to extract model configs from Provider - Support ANTHROPIC_MODEL, ANTHROPIC_REASONING_MODEL, and default models for haiku/sonnet/opus - Implement thinking mode detection for reasoning model priority - Include comprehensive unit tests for all mapping scenarios * fix(proxy): bypass circuit breaker for single provider scenario When failover is disabled (single provider), circuit breaker open state would block all requests causing poor UX. Now bypasses circuit breaker check in this scenario. Also integrates model mapping into request flow. * feat(ui): add reasoning model field to Claude provider form Add ANTHROPIC_REASONING_MODEL configuration field for Claude providers, allowing users to specify a dedicated model for thinking/reasoning tasks. * feat(proxy): add openrouter_compat_mode for optional format conversion Add configurable OpenRouter compatibility mode that enables Anthropic to OpenAI format conversion. When enabled, rewrites endpoint to /v1/chat/completions and transforms request/response formats. Defaults to enabled for OpenRouter. * feat(ui): add OpenRouter compatibility mode toggle Add UI toggle for OpenRouter providers to enable/disable compatibility mode which uses OpenAI Chat Completions format with SSE conversion. * feat(stream-check): use provider-configured model for health checks Extract model from provider's settings_config (ANTHROPIC_MODEL, GEMINI_MODEL, or Codex config.toml) instead of always using default test models. * refactor(ui): remove timeout settings from AutoFailoverConfigPanel Remove streaming/non-streaming timeout configuration from failover panel as these settings have been moved to a dedicated location. * refactor(database): migrate proxy_config to per-app three-row structure Replace singleton proxy_config table with app_type primary key structure, allowing independent proxy settings for Claude, Codex, and Gemini. Add GlobalProxyConfig queries and per-app config management in DAO layer. * feat(proxy): add GlobalProxyConfig and AppProxyConfig types Add new type definitions for the refactored proxy configuration: - GlobalProxyConfig: shared settings (enabled, address, port, logging) - AppProxyConfig: per-app settings (failover, timeouts, circuit breaker) * refactor(proxy): update service layer for per-app config structure Adapt proxy service, handler context, and provider router to use the new per-app configuration model. Read enabled/timeout settings from proxy_config table instead of settings table. * feat(commands): add global and per-app proxy config commands Add new Tauri commands for the refactored proxy configuration: - get_global_proxy_config / update_global_proxy_config - get_proxy_config_for_app / update_proxy_config_for_app Update startup restore logic to read from proxy_config table. * feat(api): add frontend API and Query hooks for proxy config Add TypeScript wrappers and TanStack Query hooks for: - Global proxy config (address, port, logging) - Per-app proxy config (failover, timeouts, circuit breaker) - Proxy takeover status management * refactor(ui): redesign proxy panel with inline config controls Replace ProxySettingsDialog with inline controls in ProxyPanel. Add per-app takeover switches and global address/port settings. Simplify AutoFailoverConfigPanel by removing timeout settings. * feat(i18n): add proxy takeover translations and update types Add i18n strings for proxy takeover status in zh/en/ja. Update TypeScript types for GlobalProxyConfig and AppProxyConfig. * refactor(proxy): load circuit breaker config per-app instead of globally Extract app_type from router key and read circuit breaker settings from the corresponding proxy_config row for each application.	2025-12-25 10:40:11 +08:00
Jason	0ef8a4153f	fix(proxy): sync UI when active provider differs from current setting ... Previously, UI sync was triggered only when failover happened (retry count > 1). This missed cases where the first provider in the failover queue succeeded but was different from the user's selected provider in settings. Now we capture the current provider ID at request start and compare it with the actually used provider. This ensures UI/tray always reflects the real provider handling requests.	2025-12-17 10:22:02 +08:00
Jason	9196d07925	feat(proxy): sync UI when failover succeeds ... Add FailoverSwitchManager to handle provider switching after successful failover. This ensures the UI reflects the actual provider in use: - Create failover_switch.rs with deduplication and async switching logic - Pass AppHandle through ProxyService -> ProxyServer -> RequestForwarder - Update is_current in database when failover succeeds - Emit provider-switched event for frontend refresh - Update tray menu and live backup synchronously The switching runs asynchronously via tokio::spawn to avoid blocking API responses while still providing immediate UI feedback.	2025-12-15 17:12:36 +08:00
Jason	5a5ca2a989	fix(proxy): resolve circuit breaker state persistence and HalfOpen deadlock ... This commit addresses several critical issues in the failover system: **Circuit breaker state persistence (previous fix)** - Promote ProviderRouter to ProxyState for cross-request state sharing - Remove redundant router.rs module - Fix 429 errors to be retryable (rate limiting should try other providers) **Hot-update circuit breaker config** - Add update_circuit_breaker_configs() to ProxyServer and ProxyService - Connect update_circuit_breaker_config command to running circuit breakers - Add reset_provider_circuit_breaker() for manual breaker reset **Fix HalfOpen deadlock bug** - Change half_open_requests from cumulative count to in-flight count - Release quota in record_success()/record_failure() when in HalfOpen state - Prevents permanent deadlock when success_threshold > 1 **Fix duplicate select_providers() call** - Store providers list in RequestContext, pass to forward_with_retry() - Avoid consuming HalfOpen quota twice per request - Single call to select_providers() per request lifecycle **Add per-provider retry with exponential backoff** - Implement forward_with_provider_retry() with configurable max_retries - Backoff delays: 100ms, 200ms, 400ms, etc.	2025-12-13 22:47:49 +08:00
Jason	ebe2a665ae	refactor(proxy): modularize handlers.rs to reduce code duplication ... Extract common request handling logic into dedicated modules: - handler_config.rs: Usage parser configurations for each API type - handler_context.rs: Request lifecycle context management - response_processor.rs: Unified streaming/non-streaming response handling Reduces handlers.rs from ~1130 lines to ~418 lines (-63%), eliminating repeated initialization and response processing patterns across the four API handlers (Claude, Codex Chat, Codex Responses, Gemini).	2025-12-11 23:22:05 +08:00