Commit Graph

6 Commits

Author SHA1 Message Date
Jason 402570ce31 fix(usage): pricing routing, SSE lifecycle, and validation hardening
* model pricing routing: extend prefix-match families (gpt-/o1-o5/
  gemini-/deepseek-/qwen-/glm-/kimi-/minimax-) with per-family dash
  thresholds so short base IDs like gpt-5 no longer mis-match
  gpt-5-mini; strip ISO and 8-digit date suffixes via UTF-8-safe
  byte matching so claude-haiku-4-5-20251001 falls back to
  claude-haiku-4-5 pricing
* SSE collector: SseUsageFinishGuard (RAII) guarantees finish() on
  early return or panic; AtomicBool fast path lets push() skip the
  Mutex once first-event time is recorded
* validation: shared validate_cost_multiplier / validate_pricing_source
  helpers across DAO and service layers; PRICING_SOURCE_RESPONSE /
  PRICING_SOURCE_REQUEST constants replace string literals; price
  fields in update_model_pricing now reject empty / non-decimal /
  negative input before INSERT
* backfill: add backfill_missing_usage_costs_for_model so a single
  price edit only scans matching rows instead of the full log table;
  startup backfill remains full-scan
* session_usage{,_codex,_gemini}: share find_model_pricing helper from
  usage_stats; metadata_modified_nanos centralizes mtime precision
* frontend: NON_NEGATIVE_DECIMAL_REGEX + isNonNegativeDecimalString
  replace three copies of the same multiplier regex; isUnpricedUsage
  surfaces zero-cost rows that have usage tokens (cached per row to
  avoid double evaluation); invalidate usageKeys.all on pricing mutate
  so backfilled rows refresh
2026-05-14 11:53:51 +08:00
Jason f061b777b7 feat(usage): add Hermes Agent tracking + fix zero-cost bug + perf
Hermes:
- Parse ~/.hermes/state.db sessions (incl. profiles/*/state.db) into
  proxy_request_logs with data_source='hermes_session', WAL-aware
  incremental sync, Hermes-reported cost preferred over model_pricing
  fallback

Zero-cost bug (dashboard showed \$0 totals):
- GPT-5.5 family default pricing (~83% of affected rows used GPT-5.5)
- find_model_pricing_row: ASCII-lowercase normalization so
  "OpenAI/GPT-5.5@HIGH" matches seeded "gpt-5.5"
- Startup cost backfill in async task: scan rows where total_cost <= 0
  but tokens > 0, recompute via model_pricing in a single transaction

Performance:
- Add (app_type, created_at DESC) covering index for dashboard range
  queries
- Add expression index on COALESCE(data_source, 'proxy') so dedup EXISTS
  subqueries use index lookup instead of full scan; drop superseded
  idx_request_logs_dedup_lookup

Refactor:
- row_to_request_log_detail helper (3-way de-dup; fixes cost_multiplier
  \"1\" vs \"1.0\" drift between callers)
- Promote get_sync_state/update_sync_state to shared session_usage
  module (4 copies -> 1)
- run_step helper in lib.rs replaces 9 if-let-Err blocks
- maybe_backfill_log_costs returns bool to skip duplicate total_cost
  parsing in caller
2026-04-29 22:41:46 +08:00
Jason 2ee7cb4101 fix(usage): prevent double-counting between proxy and session-log sources
Proxy writes and session-log sync wrote to proxy_request_logs with
mismatched request_ids: only Claude on a native Anthropic backend used the
shared `session:{message_id}` key. Codex/Gemini and Claude-through-OpenAI
providers always produced distinct ids, so primary-key dedup never fired
and every real request was recorded twice.

Adds a 7-dim fingerprint dedup (app_type, 4 token counts, 2xx status,
model with case-insensitive match, ±10min window) wired into three layers:

- Write path: should_skip_session_insert() blocks duplicate session rows
  before INSERT, unifying the previously-divergent Claude/Codex/Gemini
  paths through a single DedupKey-based helper.
- Read path: effective_usage_log_filter() excludes already-covered session
  rows from every aggregation query.
- Rollup path: same filter applied so usage_daily_rollups never absorbs
  duplicates.

Also adds a covering index (idx_request_logs_dedup_lookup) so the EXISTS
subquery stays index-only, and a transform.rs regression test that pins
openai_to_anthropic id preservation - the missing piece that lets
Claude+OpenAI-compatible providers reuse the session: id scheme.
2026-04-29 22:41:46 +08:00
Jason 8669b408e9 fix(usage): deduplicate proxy and session log usage records
Extract message_id from Claude API responses (msg_xxx) and use it to
generate a shared request_id format (session:{msg_xxx}) between the
proxy logger and session log sync. When session sync encounters the
same request_id via INSERT OR IGNORE, it skips the duplicate.

- Add message_id field to TokenUsage, extracted from Claude responses
- Add TokenUsage::dedup_request_id() to generate shared request IDs
- Define SESSION_REQUEST_ID_PREFIX constant to eliminate magic strings
- Change proxy logger to INSERT OR REPLACE for richer-data-wins semantics
2026-04-14 10:33:41 +08:00
Jason 697d0dd6e1 fix: resolve rustfmt formatting and clippy warnings
- Apply cargo fmt across schema.rs, session_usage*.rs, skill.rs, usage_stats.rs
- Fix clippy::for_kv_map: use messages.values() instead of (_, msg) pattern
- Suppress clippy::only_used_in_recursion for intentional recursive base path
- Fix prettier formatting in UsageScriptModal.tsx
2026-04-09 16:49:13 +08:00
Jason 154342ca00 feat: add session log usage tracking without proxy
Parse Claude Code JSONL session files (~/.claude/projects/) and Codex
SQLite database (~/.codex/state_5.sqlite) to track API usage without
requiring proxy interception. This enables usage statistics for users
who don't use the proxy feature.

Key changes:
- Add session_usage.rs: incremental JSONL parser with message.id dedup
- Add session_usage_codex.rs: import thread-level token data from Codex
- Add data_source column to proxy_request_logs (proxy/session_log/codex_db)
- Add session_log_sync table for tracking parse offsets
- Background sync every 60s + manual sync via DataSourceBar UI
- Schema migration v7→v8
- i18n support for zh/en/ja
2026-04-09 16:49:13 +08:00