Feat/usage improvements (#508)

* i18n: update cache terminology across all languages

- Change 'Cache Read' to 'Cache Hit' in all languages
- Change 'Cache Write' to 'Cache Creation' in all languages
- Update zh: 缓存读取 → 缓存命中, 缓存写入 → 缓存创建
- Update en: Cache Read → Cache Hit, Cache Write → Cache Creation
- Update ja: キャッシュ読取 → キャッシュヒット, キャッシュ書込 → キャッシュ作成

Affected keys: cacheReadTokens, cacheCreationTokens, cacheReadCost,
cacheWriteCost, cacheRead, cacheWrite

* feat(usage): add cache metrics to trend chart

- Add cache creation tokens visualization (orange line)
- Add cache hit tokens visualization (purple line)
- Add gradient definitions for new cache metrics
- Include cache data in hourly aggregation
- Display cache metrics alongside input/output tokens

This provides better visibility into cache usage patterns over time.

* fix(usage): fix timezone handling in datetime picker

- Add timestampToLocalDatetime() to convert Unix timestamp to local datetime
- Add localDatetimeToTimestamp() with validation for incomplete input
- Fix issue where typing hours/minutes would jump to previous day
- Validate datetime format completeness before conversion
- Use local timezone instead of UTC for datetime-local input

This resolves the issue where users couldn't fine-tune time selection
and the input would jump unexpectedly when editing hours or minutes.

* feat(usage): add auto-refresh for usage statistics

- Add 30-second auto-refresh interval for all usage queries
- Disable background refresh to save resources
- Apply to: summary, trends, provider stats, model stats, request logs
- Queries automatically update when tab is active
- Pause refresh when user switches to another tab

This keeps usage data fresh without manual refresh.

* fix(proxy): improve usage logging and cache token parsing

- Log requests even when usage parsing fails (with default values)
- Add detailed debug logging for usage metrics
- Support cache_read_input_tokens field in Codex responses
- Fallback to input_tokens_details.cached_tokens if needed
- Add test case for cached_tokens in input_tokens_details
- Ensure all requests are tracked in database for analytics

This fixes missing request logs when API responses lack usage data
and improves cache token detection across different response formats.

* style(rust): use inline format args in format! macros

- Replace format!("...", var) with format!("...{var}")
- Update universal provider ID formatting
- Update error message formatting
- Update config.toml generation in Codex provider

Fixes clippy::uninlined_format_args warnings.

* feat(proxy): enhance provider router logging

- Add debug logs for failover queue provider count
- Log circuit breaker state for each provider check
- Add logs for missing current provider scenarios
- Log when no current provider is configured
- Use inline format args for better readability

This improves debugging of provider selection and failover behavior.

* feat(database): update model pricing data

- Update Claude models to full version format (e.g. claude-opus-4-5-20251101)
- Add GPT-5.2 series model pricing (10 models)
- Add GPT-5.1 series model pricing (10 models)
- Add GPT-5 series model pricing (12 models)
- Add Gemini 3 series model pricing (2 models)
- Update Gemini 2.5 series model ID format (use dot separator)
- Unify display names by removing thinking level suffixes

* fix(usage): correct Gemini output token calculation

Fix Gemini API output token parsing to use totalTokenCount - promptTokenCount
instead of candidatesTokenCount alone. This ensures thoughtsTokenCount is
included in output statistics.

- Update from_gemini_response to calculate output from total - input
- Update from_gemini_stream_chunks with same logic for consistency
- Fix from_codex_stream_events to use adjusted token calculation
- Add test case for responses with thoughtsTokenCount
- Update existing tests to match new calculation logic

* fix(usage): correct cache token billing and add Codex format auto-detection

- Avoid double-billing cache tokens by subtracting from input before calculation
- Add smart Codex parser that auto-detects OpenAI vs Codex API format
- Extract model name from Codex responses for accurate tracking

* fix(proxy): improve takeover detection with live config check

- Add live config takeover detection for hot-switch decision
- Rebuild takeover when backup is missing or placeholder remains
- Make detect_takeover_in_live_config_for_app public
- Fix is_takeover_active to use actual takeover status

* refactor(usage): simplify model pricing lookup by removing suffix fallback

Replace complex suffix-stripping fallback with direct prefix/suffix cleanup.
Model IDs are now cleaned by removing vendor prefix (before /) and colon
suffix (after :), then matched exactly against pricing table.

* feat(database): add Chinese AI model pricing data

Add pricing for domestic AI models (CNY/1M tokens):
- Doubao-Seed-Code (ByteDance)
- DeepSeek V3/V3.1/V3.2
- Kimi K2/K2-Thinking/K2-Turbo (Moonshot)
- MiniMax M2/M2.1/M2.1-Lightning
- GLM-4.6/4.7 (Zhipu)
- Mimo V2 Flash (Xiaomi)

Also fix test case to use correct model ID and remove invalid currency column.

* refactor(proxy): improve header forwarding with blacklist approach

Change from whitelist to blacklist mode for request header forwarding.
Only skip headers that will be overridden (auth, host, content-length).
This preserves client's original headers and improves compatibility.

* fix(proxy): bypass timeout and retry configs when failover is disabled

When auto_failover_enabled is false, timeout and retry configurations
should not affect normal request flow. This change ensures:

- create_forwarder: passes 0 for all timeout/retry params when failover
  is disabled, effectively bypassing these checks
- streaming_timeout_config: returns 0 for both first_byte_timeout and
  idle_timeout when failover is disabled

This prevents unnecessary timeout errors and retry attempts when users
have explicitly disabled the failover feature.

* fix(proxy): handle zero value input in failover config fields

* refactor(proxy): remove retry logic and add enabled check for failover

* refactor(proxy): distinguish circuit-open from no-provider errors

* Align usage stats to sliding windows

* feat(proxy): add body and header filtering for upstream requests

* feat(proxy): enable transparent passthrough for headers

- Passthrough anthropic-beta header as-is from client
- Passthrough anthropic-version header from client
- Passthrough client IP headers (x-forwarded-for, x-real-ip) by default
- Filter private params (underscore-prefixed fields) from request body
- No database changes required

* feat(proxy): extract session ID from client requests for logging

- Add SessionIdExtractor to parse session ID from Claude/Codex requests
- Support extraction from metadata.user_id, headers, previous_response_id
- Pass session_id through RequestContext to usage logger
- Enable request correlation by session in proxy_request_logs
This commit is contained in:
Dex Miller
2025-12-31 22:57:00 +08:00
committed by GitHub
parent d0431b66ae
commit 5376ea042b
31 changed files with 1888 additions and 583 deletions
+187 -97
View File
@@ -1,8 +1,9 @@
//! 请求转发器
//!
//! 负责将请求转发到上游Provider,支持重试和故障转移
//! 负责将请求转发到上游Provider,支持故障转移
use super::{
body_filter::filter_private_params_with_whitelist,
error::*,
failover_switch::FailoverSwitchManager,
provider_router::ProviderRouter,
@@ -17,6 +18,71 @@ use std::sync::Arc;
use std::time::{Duration, Instant};
use tokio::sync::RwLock;
/// Headers 黑名单 - 不透传到上游的 Headers
///
/// 参考 Claude Code Hub 设计,过滤以下类别:
/// 1. 认证类(会被覆盖)
/// 2. 连接类(由 HTTP 客户端管理)
/// 3. 代理转发类
/// 4. CDN/云服务商特定头
/// 5. 请求追踪类
/// 6. 浏览器特定头(可能被上游检测)
///
/// 注意:客户端 IP 类(x-forwarded-for, x-real-ip)默认透传
const HEADER_BLACKLIST: &[&str] = &[
// 认证类(会被覆盖)
"authorization",
"x-api-key",
// 连接类
"host",
"content-length",
"connection",
"transfer-encoding",
// 编码类(会被覆盖为 identity)
"accept-encoding",
// 代理转发类(保留 x-forwarded-for 和 x-real-ip
"x-forwarded-host",
"x-forwarded-port",
"x-forwarded-proto",
"forwarded",
// CDN/云服务商特定头
"cf-connecting-ip",
"cf-ipcountry",
"cf-ray",
"cf-visitor",
"true-client-ip",
"fastly-client-ip",
"x-azure-clientip",
"x-azure-fdid",
"x-azure-ref",
"akamai-origin-hop",
"x-akamai-config-log-detail",
// 请求追踪类
"x-request-id",
"x-correlation-id",
"x-trace-id",
"x-amzn-trace-id",
"x-b3-traceid",
"x-b3-spanid",
"x-b3-parentspanid",
"x-b3-sampled",
"traceparent",
"tracestate",
// 浏览器特定头(可能被上游检测为非 CLI 请求)
"sec-fetch-mode",
"sec-fetch-site",
"sec-fetch-dest",
"sec-ch-ua",
"sec-ch-ua-mobile",
"sec-ch-ua-platform",
"accept-language",
// anthropic-beta 单独处理,避免重复
"anthropic-beta",
// 客户端 IP 单独处理(默认透传)
"x-forwarded-for",
"x-real-ip",
];
pub struct ForwardResult {
pub response: Response,
pub provider: Provider,
@@ -31,8 +97,6 @@ pub struct RequestForwarder {
client: Client,
/// 共享的 ProviderRouter(持有熔断器状态)
router: Arc<ProviderRouter>,
/// 单个 Provider 内的最大重试次数
max_retries: u8,
status: Arc<RwLock<ProxyStatus>>,
current_providers: Arc<RwLock<std::collections::HashMap<String, (String, String)>>>,
/// 故障转移切换管理器
@@ -48,7 +112,6 @@ impl RequestForwarder {
pub fn new(
router: Arc<ProviderRouter>,
non_streaming_timeout: u64,
max_retries: u8,
status: Arc<RwLock<ProxyStatus>>,
current_providers: Arc<RwLock<std::collections::HashMap<String, (String, String)>>>,
failover_manager: Arc<FailoverSwitchManager>,
@@ -77,7 +140,6 @@ impl RequestForwarder {
Self {
client,
router,
max_retries,
status,
current_providers,
failover_manager,
@@ -86,59 +148,6 @@ impl RequestForwarder {
}
}
/// 对单个 Provider 执行请求(带重试)
///
/// 在同一个 Provider 上最多重试 max_retries 次,使用指数退避
async fn forward_with_provider_retry(
&self,
provider: &Provider,
endpoint: &str,
body: &Value,
headers: &axum::http::HeaderMap,
adapter: &dyn ProviderAdapter,
) -> Result<Response, ProxyError> {
let mut last_error = None;
for attempt in 0..=self.max_retries {
if attempt > 0 {
// 指数退避:100ms, 200ms, 400ms, ...
let delay_ms = 100 * 2u64.pow(attempt as u32 - 1);
log::info!(
"[{}] 重试第 {}/{} 次(等待 {}ms",
adapter.name(),
attempt,
self.max_retries,
delay_ms
);
tokio::time::sleep(Duration::from_millis(delay_ms)).await;
}
match self
.forward(provider, endpoint, body, headers, adapter)
.await
{
Ok(response) => return Ok(response),
Err(e) => {
// 只有“同一 Provider 内可重试”的错误才继续重试
if !self.should_retry_same_provider(&e) {
return Err(e);
}
log::debug!(
"[{}] Provider {} 第 {} 次请求失败: {}",
adapter.name(),
provider.name,
attempt + 1,
e
);
last_error = Some(e);
}
}
}
Err(last_error.unwrap_or(ProxyError::MaxRetriesExceeded))
}
/// 转发请求(带故障转移)
///
/// # Arguments
@@ -224,9 +233,9 @@ impl RequestForwarder {
let start = Instant::now();
// 转发请求(带单 Provider 内重试
// 转发请求(每个 Provider 只尝试一次,重试由客户端控制
match self
.forward_with_provider_retry(provider, endpoint, &body, &headers, adapter.as_ref())
.forward(provider, endpoint, &body, &headers, adapter.as_ref())
.await
{
Ok(response) => {
@@ -477,6 +486,28 @@ impl RequestForwarder {
mapped_body
};
// 过滤私有参数(以 `_` 开头的字段),防止内部信息泄露到上游
// 默认使用空白名单,过滤所有 _ 前缀字段
let filtered_body = filter_private_params_with_whitelist(request_body, &[]);
// ========== 请求体日志(截断显示) ==========
let body_str = serde_json::to_string_pretty(&filtered_body)
.unwrap_or_else(|_| filtered_body.to_string());
let body_preview = if body_str.len() > 2000 {
format!(
"{}...\n[截断,总长度: {} 字符]",
&body_str[..2000],
body_str.len()
)
} else {
body_str
};
log::info!(
"[{}] ====== 最终请求体 ======\n{}",
adapter.name(),
body_preview
);
log::info!(
"[{}] 转发请求: {} -> {}",
adapter.name(),
@@ -487,28 +518,73 @@ impl RequestForwarder {
// 构建请求
let mut request = self.client.post(&url);
// 只透传必要的 Headers(白名单模式)
let allowed_headers = [
"accept",
"user-agent",
"x-request-id",
"x-stainless-arch",
"x-stainless-lang",
"x-stainless-os",
"x-stainless-package-version",
"x-stainless-runtime",
"x-stainless-runtime-version",
];
// ========== 详细 Headers 日志 ==========
log::info!("[{}] ====== 客户端原始 Headers ======", adapter.name());
for (key, value) in headers {
log::info!(
"[{}] {}: {:?}",
adapter.name(),
key.as_str(),
value.to_str().unwrap_or("<binary>")
);
}
// 过滤黑名单 Headers,保护隐私并避免冲突
let mut filtered_headers: Vec<String> = Vec::new();
let mut passed_headers: Vec<(String, String)> = Vec::new();
for (key, value) in headers {
let key_str = key.as_str().to_lowercase();
if allowed_headers.contains(&key_str.as_str()) {
request = request.header(key, value);
if HEADER_BLACKLIST.contains(&key_str.as_str()) {
filtered_headers.push(key_str);
continue;
}
let value_str = value.to_str().unwrap_or("<binary>").to_string();
passed_headers.push((key.as_str().to_string(), value_str.clone()));
request = request.header(key, value);
}
if !filtered_headers.is_empty() {
log::info!(
"[{}] ====== 被过滤的 Headers ({}) ======",
adapter.name(),
filtered_headers.len()
);
for h in &filtered_headers {
log::info!("[{}] - {}", adapter.name(), h);
}
}
// 确保 Content-Type 是 json
request = request.header("Content-Type", "application/json");
// 处理 anthropic-beta Header(透传)
// 参考 Claude Code Hub 的实现,直接透传客户端的 beta 标记
if let Some(beta) = headers.get("anthropic-beta") {
if let Ok(beta_str) = beta.to_str() {
request = request.header("anthropic-beta", beta_str);
passed_headers.push(("anthropic-beta".to_string(), beta_str.to_string()));
log::info!("[{}] 透传 anthropic-beta: {}", adapter.name(), beta_str);
}
}
// 客户端 IP 透传(默认开启)
if let Some(xff) = headers.get("x-forwarded-for") {
if let Ok(xff_str) = xff.to_str() {
request = request.header("x-forwarded-for", xff_str);
passed_headers.push(("x-forwarded-for".to_string(), xff_str.to_string()));
log::debug!("[{}] 透传 x-forwarded-for: {}", adapter.name(), xff_str);
}
}
if let Some(real_ip) = headers.get("x-real-ip") {
if let Ok(real_ip_str) = real_ip.to_str() {
request = request.header("x-real-ip", real_ip_str);
passed_headers.push(("x-real-ip".to_string(), real_ip_str.to_string()));
log::debug!("[{}] 透传 x-real-ip: {}", adapter.name(), real_ip_str);
}
}
// 禁用压缩,避免 gzip 流式响应解析错误
// 参考 CCH: undici 在连接提前关闭时会对不完整的 gzip 流抛出错误
request = request.header("accept-encoding", "identity");
passed_headers.push(("accept-encoding".to_string(), "identity".to_string()));
// 使用适配器添加认证头
if let Some(auth) = adapter.extract_auth(provider) {
@@ -519,6 +595,15 @@ impl RequestForwarder {
auth.masked_key()
);
request = adapter.add_auth_headers(request, &auth);
// 记录认证头(脱敏)
passed_headers.push((
"authorization".to_string(),
format!("Bearer {}...", &auth.api_key[..8.min(auth.api_key.len())]),
));
passed_headers.push((
"x-api-key".to_string(),
format!("{}...", &auth.api_key[..8.min(auth.api_key.len())]),
));
} else {
log::error!(
"[{}] 未找到 API KeyProvider: {}",
@@ -527,9 +612,34 @@ impl RequestForwarder {
);
}
// anthropic-version 透传:优先使用客户端的版本号
// 参考 Claude Code Hub:透传客户端值而非固定版本
if let Some(version) = headers.get("anthropic-version") {
if let Ok(version_str) = version.to_str() {
// 覆盖适配器设置的默认版本
request = request.header("anthropic-version", version_str);
passed_headers.push(("anthropic-version".to_string(), version_str.to_string()));
log::info!(
"[{}] 透传 anthropic-version: {}",
adapter.name(),
version_str
);
}
}
// ========== 最终发送的 Headers 日志 ==========
log::info!(
"[{}] ====== 最终发送的 Headers ({}) ======",
adapter.name(),
passed_headers.len()
);
for (k, v) in &passed_headers {
log::info!("[{}] {}: {}", adapter.name(), k, v);
}
// 发送请求
log::info!("[{}] 发送请求到: {}", adapter.name(), url);
let response = request.json(&request_body).send().await.map_err(|e| {
let response = request.json(&filtered_body).send().await.map_err(|e| {
log::error!("[{}] 请求失败: {}", adapter.name(), e);
if e.is_timeout() {
ProxyError::Timeout(format!("请求超时: {e}"))
@@ -563,25 +673,6 @@ impl RequestForwarder {
}
}
/// 分类ProxyError
///
/// 决定哪些错误应该触发故障转移到下一个 Provider
///
/// 设计原则:既然用户配置了多个供应商,就应该让所有供应商都尝试一遍。
/// 只有明确是客户端中断的情况才不重试。
fn should_retry_same_provider(&self, error: &ProxyError) -> bool {
match error {
// 网络类错误:短暂抖动时同一 Provider 内重试有意义
ProxyError::Timeout(_) => true,
ProxyError::ForwardFailed(_) => true,
// 上游 HTTP 错误:只对“可能瞬态”的状态码做同 Provider 重试(其余交给 failover
ProxyError::UpstreamError { status, .. } => {
*status == 408 || *status == 429 || *status >= 500
}
_ => false,
}
}
fn categorize_proxy_error(&self, error: &ProxyError) -> ErrorCategory {
match error {
// 网络和上游错误:都应该尝试下一个供应商
@@ -597,7 +688,6 @@ impl RequestForwarder {
ProxyError::TransformError(_) => ErrorCategory::Retryable,
ProxyError::AuthError(_) => ErrorCategory::Retryable,
ProxyError::StreamIdleTimeout(_) => ErrorCategory::Retryable,
ProxyError::MaxRetriesExceeded => ErrorCategory::Retryable,
// 无可用供应商:所有供应商都试过了,无法重试
ProxyError::NoAvailableProvider => ErrorCategory::NonRetryable,
// 其他错误(数据库/内部错误等):不是换供应商能解决的问题