Files
CC-Switch/src-tauri/src/proxy/handler_context.rs
T
Dex Miller 5376ea042b Feat/usage improvements (#508)
* i18n: update cache terminology across all languages

- Change 'Cache Read' to 'Cache Hit' in all languages
- Change 'Cache Write' to 'Cache Creation' in all languages
- Update zh: 缓存读取 → 缓存命中, 缓存写入 → 缓存创建
- Update en: Cache Read → Cache Hit, Cache Write → Cache Creation
- Update ja: キャッシュ読取 → キャッシュヒット, キャッシュ書込 → キャッシュ作成

Affected keys: cacheReadTokens, cacheCreationTokens, cacheReadCost,
cacheWriteCost, cacheRead, cacheWrite

* feat(usage): add cache metrics to trend chart

- Add cache creation tokens visualization (orange line)
- Add cache hit tokens visualization (purple line)
- Add gradient definitions for new cache metrics
- Include cache data in hourly aggregation
- Display cache metrics alongside input/output tokens

This provides better visibility into cache usage patterns over time.

* fix(usage): fix timezone handling in datetime picker

- Add timestampToLocalDatetime() to convert Unix timestamp to local datetime
- Add localDatetimeToTimestamp() with validation for incomplete input
- Fix issue where typing hours/minutes would jump to previous day
- Validate datetime format completeness before conversion
- Use local timezone instead of UTC for datetime-local input

This resolves the issue where users couldn't fine-tune time selection
and the input would jump unexpectedly when editing hours or minutes.

* feat(usage): add auto-refresh for usage statistics

- Add 30-second auto-refresh interval for all usage queries
- Disable background refresh to save resources
- Apply to: summary, trends, provider stats, model stats, request logs
- Queries automatically update when tab is active
- Pause refresh when user switches to another tab

This keeps usage data fresh without manual refresh.

* fix(proxy): improve usage logging and cache token parsing

- Log requests even when usage parsing fails (with default values)
- Add detailed debug logging for usage metrics
- Support cache_read_input_tokens field in Codex responses
- Fallback to input_tokens_details.cached_tokens if needed
- Add test case for cached_tokens in input_tokens_details
- Ensure all requests are tracked in database for analytics

This fixes missing request logs when API responses lack usage data
and improves cache token detection across different response formats.

* style(rust): use inline format args in format! macros

- Replace format!("...", var) with format!("...{var}")
- Update universal provider ID formatting
- Update error message formatting
- Update config.toml generation in Codex provider

Fixes clippy::uninlined_format_args warnings.

* feat(proxy): enhance provider router logging

- Add debug logs for failover queue provider count
- Log circuit breaker state for each provider check
- Add logs for missing current provider scenarios
- Log when no current provider is configured
- Use inline format args for better readability

This improves debugging of provider selection and failover behavior.

* feat(database): update model pricing data

- Update Claude models to full version format (e.g. claude-opus-4-5-20251101)
- Add GPT-5.2 series model pricing (10 models)
- Add GPT-5.1 series model pricing (10 models)
- Add GPT-5 series model pricing (12 models)
- Add Gemini 3 series model pricing (2 models)
- Update Gemini 2.5 series model ID format (use dot separator)
- Unify display names by removing thinking level suffixes

* fix(usage): correct Gemini output token calculation

Fix Gemini API output token parsing to use totalTokenCount - promptTokenCount
instead of candidatesTokenCount alone. This ensures thoughtsTokenCount is
included in output statistics.

- Update from_gemini_response to calculate output from total - input
- Update from_gemini_stream_chunks with same logic for consistency
- Fix from_codex_stream_events to use adjusted token calculation
- Add test case for responses with thoughtsTokenCount
- Update existing tests to match new calculation logic

* fix(usage): correct cache token billing and add Codex format auto-detection

- Avoid double-billing cache tokens by subtracting from input before calculation
- Add smart Codex parser that auto-detects OpenAI vs Codex API format
- Extract model name from Codex responses for accurate tracking

* fix(proxy): improve takeover detection with live config check

- Add live config takeover detection for hot-switch decision
- Rebuild takeover when backup is missing or placeholder remains
- Make detect_takeover_in_live_config_for_app public
- Fix is_takeover_active to use actual takeover status

* refactor(usage): simplify model pricing lookup by removing suffix fallback

Replace complex suffix-stripping fallback with direct prefix/suffix cleanup.
Model IDs are now cleaned by removing vendor prefix (before /) and colon
suffix (after :), then matched exactly against pricing table.

* feat(database): add Chinese AI model pricing data

Add pricing for domestic AI models (CNY/1M tokens):
- Doubao-Seed-Code (ByteDance)
- DeepSeek V3/V3.1/V3.2
- Kimi K2/K2-Thinking/K2-Turbo (Moonshot)
- MiniMax M2/M2.1/M2.1-Lightning
- GLM-4.6/4.7 (Zhipu)
- Mimo V2 Flash (Xiaomi)

Also fix test case to use correct model ID and remove invalid currency column.

* refactor(proxy): improve header forwarding with blacklist approach

Change from whitelist to blacklist mode for request header forwarding.
Only skip headers that will be overridden (auth, host, content-length).
This preserves client's original headers and improves compatibility.

* fix(proxy): bypass timeout and retry configs when failover is disabled

When auto_failover_enabled is false, timeout and retry configurations
should not affect normal request flow. This change ensures:

- create_forwarder: passes 0 for all timeout/retry params when failover
  is disabled, effectively bypassing these checks
- streaming_timeout_config: returns 0 for both first_byte_timeout and
  idle_timeout when failover is disabled

This prevents unnecessary timeout errors and retry attempts when users
have explicitly disabled the failover feature.

* fix(proxy): handle zero value input in failover config fields

* refactor(proxy): remove retry logic and add enabled check for failover

* refactor(proxy): distinguish circuit-open from no-provider errors

* Align usage stats to sliding windows

* feat(proxy): add body and header filtering for upstream requests

* feat(proxy): enable transparent passthrough for headers

- Passthrough anthropic-beta header as-is from client
- Passthrough anthropic-version header from client
- Passthrough client IP headers (x-forwarded-for, x-real-ip) by default
- Filter private params (underscore-prefixed fields) from request body
- No database changes required

* feat(proxy): extract session ID from client requests for logging

- Add SessionIdExtractor to parse session ID from Claude/Codex requests
- Support extraction from metadata.user_id, headers, previous_response_id
- Pass session_id through RequestContext to usage logger
- Enable request correlation by session in proxy_request_logs
2025-12-31 22:57:00 +08:00

248 lines
8.4 KiB
Rust
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
//! 请求上下文模块
//!
//! 提供请求生命周期的上下文管理,封装通用初始化逻辑
use crate::app_config::AppType;
use crate::provider::Provider;
use crate::proxy::{
extract_session_id, forwarder::RequestForwarder, server::ProxyState, types::AppProxyConfig,
ProxyError,
};
use axum::http::HeaderMap;
use std::time::Instant;
/// 流式超时配置
#[derive(Debug, Clone, Copy)]
pub struct StreamingTimeoutConfig {
/// 首字节超时(秒),0 表示禁用
pub first_byte_timeout: u64,
/// 静默期超时(秒),0 表示禁用
pub idle_timeout: u64,
}
/// 请求上下文
///
/// 贯穿整个请求生命周期,包含:
/// - 计时信息
/// - 应用级代理配置(per-app
/// - 选中的 Provider 列表(用于故障转移)
/// - 请求模型名称
/// - 日志标签
/// - Session ID(用于日志关联)
pub struct RequestContext {
/// 请求开始时间
pub start_time: Instant,
/// 应用级代理配置(per-app,包含重试次数和超时配置)
pub app_config: AppProxyConfig,
/// 选中的 Provider(故障转移链的第一个)
pub provider: Provider,
/// 完整的 Provider 列表(用于故障转移)
providers: Vec<Provider>,
/// 请求开始时的"当前供应商"(用于判断是否需要同步 UI/托盘)
///
/// 这里使用本地 settings 的设备级 current provider。
/// 代理模式下如果实际使用的 provider 与此不一致,会触发切换以确保 UI 始终准确。
pub current_provider_id: String,
/// 请求中的模型名称
pub request_model: String,
/// 日志标签(如 "Claude"、"Codex"、"Gemini"
pub tag: &'static str,
/// 应用类型字符串(如 "claude"、"codex"、"gemini"
pub app_type_str: &'static str,
/// 应用类型(预留,目前通过 app_type_str 使用)
#[allow(dead_code)]
pub app_type: AppType,
/// Session ID(从客户端请求提取或新生成)
pub session_id: String,
}
impl RequestContext {
/// 创建请求上下文
///
/// # Arguments
/// * `state` - 代理服务器状态
/// * `body` - 请求体 JSON
/// * `headers` - 请求头(用于提取 Session ID
/// * `app_type` - 应用类型
/// * `tag` - 日志标签
/// * `app_type_str` - 应用类型字符串
///
/// # Errors
/// 返回 `ProxyError` 如果 Provider 选择失败
pub async fn new(
state: &ProxyState,
body: &serde_json::Value,
headers: &HeaderMap,
app_type: AppType,
tag: &'static str,
app_type_str: &'static str,
) -> Result<Self, ProxyError> {
let start_time = Instant::now();
// 从数据库读取应用级代理配置(per-app)
let app_config = state
.db
.get_proxy_config_for_app(app_type_str)
.await
.map_err(|e| ProxyError::DatabaseError(e.to_string()))?;
let current_provider_id =
crate::settings::get_current_provider(&app_type).unwrap_or_default();
// 从请求体提取模型名称
let request_model = body
.get("model")
.and_then(|m| m.as_str())
.unwrap_or("unknown")
.to_string();
// 提取 Session ID
let session_result = extract_session_id(headers, body, app_type_str);
let session_id = session_result.session_id.clone();
log::debug!(
"[{}] Session ID: {} (from {:?}, client_provided: {})",
tag,
session_id,
session_result.source,
session_result.client_provided
);
// 使用共享的 ProviderRouter 选择 Provider(熔断器状态跨请求保持)
// 注意:只在这里调用一次,结果传递给 forwarder,避免重复消耗 HalfOpen 名额
let providers = state
.provider_router
.select_providers(app_type_str)
.await
.map_err(|e| match e {
crate::error::AppError::AllProvidersCircuitOpen => {
ProxyError::AllProvidersCircuitOpen
}
crate::error::AppError::NoProvidersConfigured => ProxyError::NoProvidersConfigured,
_ => ProxyError::DatabaseError(e.to_string()),
})?;
let provider = providers
.first()
.cloned()
.ok_or(ProxyError::NoAvailableProvider)?;
log::info!(
"[{}] Provider: {}, model: {}, failover chain: {} providers, session: {}",
tag,
provider.name,
request_model,
providers.len(),
session_id
);
Ok(Self {
start_time,
app_config,
provider,
providers,
current_provider_id,
request_model,
tag,
app_type_str,
app_type,
session_id,
})
}
/// 从 URI 提取模型名称(Gemini 专用)
///
/// Gemini API 的模型名称在 URI 中,格式如:
/// `/v1beta/models/gemini-pro:generateContent`
pub fn with_model_from_uri(mut self, uri: &axum::http::Uri) -> Self {
let endpoint = uri
.path_and_query()
.map(|pq| pq.as_str())
.unwrap_or(uri.path());
self.request_model = endpoint
.split('/')
.find(|s| s.starts_with("models/"))
.and_then(|s| s.strip_prefix("models/"))
.map(|s| s.split(':').next().unwrap_or(s))
.unwrap_or("unknown")
.to_string();
log::info!("[{}] 从 URI 提取模型: {}", self.tag, self.request_model);
self
}
/// 创建 RequestForwarder
///
/// 使用共享的 ProviderRouter,确保熔断器状态跨请求保持
///
/// 配置生效规则:
/// - 故障转移开启:超时配置正常生效(0 表示禁用超时)
/// - 故障转移关闭:超时配置不生效(全部传入 0)
pub fn create_forwarder(&self, state: &ProxyState) -> RequestForwarder {
let (non_streaming_timeout, first_byte_timeout, idle_timeout) =
if self.app_config.auto_failover_enabled {
// 故障转移开启:使用配置的值(0 = 禁用超时)
(
self.app_config.non_streaming_timeout as u64,
self.app_config.streaming_first_byte_timeout as u64,
self.app_config.streaming_idle_timeout as u64,
)
} else {
// 故障转移关闭:不启用超时配置
log::info!(
"[{}] Failover disabled, timeout configs are bypassed",
self.tag
);
(0, 0, 0)
};
RequestForwarder::new(
state.provider_router.clone(),
non_streaming_timeout,
state.status.clone(),
state.current_providers.clone(),
state.failover_manager.clone(),
state.app_handle.clone(),
self.current_provider_id.clone(),
first_byte_timeout,
idle_timeout,
)
}
/// 获取 Provider 列表(用于故障转移)
///
/// 返回在创建上下文时已选择的 providers,避免重复调用 select_providers()
pub fn get_providers(&self) -> Vec<Provider> {
self.providers.clone()
}
/// 计算请求延迟(毫秒)
#[inline]
pub fn latency_ms(&self) -> u64 {
self.start_time.elapsed().as_millis() as u64
}
/// 获取流式超时配置
///
/// 配置生效规则:
/// - 故障转移开启:返回配置的值(0 表示禁用超时检查)
/// - 故障转移关闭:返回 0(禁用超时检查)
#[inline]
pub fn streaming_timeout_config(&self) -> StreamingTimeoutConfig {
if self.app_config.auto_failover_enabled {
// 故障转移开启:使用配置的值(0 = 禁用超时)
StreamingTimeoutConfig {
first_byte_timeout: self.app_config.streaming_first_byte_timeout as u64,
idle_timeout: self.app_config.streaming_idle_timeout as u64,
}
} else {
// 故障转移关闭:禁用流式超时检查
StreamingTimeoutConfig {
first_byte_timeout: 0,
idle_timeout: 0,
}
}
}
}