Commit Graph

11 Commits

Author SHA1 Message Date
Jason 13e7c1fcc4 fix(usage): account for Anthropic cache write TTLs
Parse and retain Anthropic's ephemeral 5-minute and 1-hour cache-creation token buckets while preserving the existing aggregate cache-write metric for compatibility.

Price 1-hour writes at the documented premium relative to the configured 5-minute write rate, clamp inconsistent provider details safely, and include TTL buckets in usage diagnostics.

Persist 1-hour cache-write tokens with schema version 14 so zero-cost backfills and later pricing updates retain the original TTL semantics. Keep session import paths compatible through zero-valued detail fields.
2026-07-13 16:48:21 +08:00
Jason 05bc14e82b fix(usage): import billable session messages without stop_reason
The local session-log scanner dropped any assistant message that lacked
a stop_reason or had output_tokens==0. Claude Code Workflow / sub-agent
fan-out frequently produces messages that only wrote a message_start
snapshot (output=1, stop_reason=None) without a final block, yet their
input + cache_read + cache_creation tokens are already billed by
Anthropic (charged once the request is accepted). Dropping them
under-counted usage by ~4.1% overall, 92% concentrated in
workflow/subagent transcripts.

Replace the stop_reason/output gate with a billable-token check (any of
input/output/cache_read/cache_creation > 0). The per-message-id dedup
selection is unchanged, and request_id = "session:"+msg_id PRIMARY KEY
with INSERT OR IGNORE keeps each message single-inserted, so relaxing
the gate cannot double-count. Add a regression test covering a
stop_reason-less message with real cache cost plus an all-zero skip.

This is the parser-layer half of the Workflow under-counting fixed at
the collector layer in 8d332925.
2026-06-11 08:29:29 +08:00
Jason 0396cd5491 fix(usage): count Claude Code Workflow sub-agent token usage
collect_jsonl_files only walked <project>/<session>/subagents/*.jsonl,
so it missed Workflow sub-agent transcripts which live one level deeper
at subagents/workflows/wf_*/agent-*.jsonl. As a result all Workflow
token usage was invisible to the no-proxy session-log accounting.

Descend into subagents/workflows/wf_*/ as well, via a new
push_jsonl_children helper that keeps the fixed-depth, no-recursion
design. journal.jsonl carries no assistant rows so it is skipped at
parse time and needs no filename special-casing. Existing dedup
(request_id PK + INSERT OR IGNORE + should_skip_session_insert) keeps
the next sync's backfill idempotent.

Add test_collect_jsonl_files_includes_workflow_subagents.
2026-06-11 08:29:29 +08:00
in30mn1a bc1467db8d feat(usage): real-time stats refresh + fix codex sync panic on non-ASCII model names (#3027)
The usage dashboard previously only refreshed on app restart for users
who don't route through the cc-switch proxy. Two issues were involved:

1. The session-sync background task panicked when a Codex model name
   contained non-ASCII characters (e.g. `【官】glm-5.1`), because
   `normalize_codex_model` sliced `&name[name.len() - 11..]` without
   verifying char boundaries. Once the task panicked, no session logs
   were imported until the app was restarted (where startup-time
   `rollup_and_prune` happened to flush pending data).

2. Even with sync working, the dashboard only polled every 30s and
   skipped polling when the window was unfocused, so freshly-imported
   data was invisible until the next poll or window refocus.

Fixes
-----

* `normalize_codex_model`: guard the 11-byte ISO-date suffix slice with
  `is_char_boundary` + `is_ascii` checks. ASCII-only suffix means the
  date-stripping logic is correct, and non-ASCII names (which can never
  be valid date suffixes anyway) now bypass the slice safely.

* New `usage_events` module that emits `usage-log-recorded` to the
  frontend whenever `proxy_request_logs` actually gains a new row.
  Sources covered: proxy `log_request`, Claude/Codex/Gemini session
  sync, and startup `rollup_and_prune`. Notifications use a global
  `OnceLock<AppHandle>` so call sites that don't already hold an
  `AppHandle` (e.g. `UsageLogger`) can notify without signature churn.

* 200ms debounce in `notify_log_recorded` collapses bursts (a single
  Codex sync importing 3000+ entries triggers ~2 emits, not 3000) so
  the frontend's `invalidateQueries` is never spammed.

* Frontend `useUsageEventBridge` listens for the event and invalidates
  `usageKeys.all`. Hook is mounted only on `UsageDashboard`, so the
  listener is unsubscribed automatically when the user navigates away.

Verification
------------

* `cargo check` passes (existing 25 dead-code warnings in
  `commands/misc.rs` are pre-existing and unrelated).
* `tsc --noEmit` passes.
* Manually verified end-to-end: a Codex sync run that imported 3145
  entries produced 2 debounced emits, both logged as `emit
  usage-log-recorded 成功`, and the dashboard updated within ~200ms.

Behaviour notes
---------------

* `INSERT OR IGNORE` paths (Claude/Codex session sync) only notify when
  the row is actually inserted, so dedup-skipped writes don't trigger
  empty refreshes.
* Gemini's `INSERT … ON CONFLICT … DO UPDATE` path reuses the existing
  `conn.changes() > 0` check and only notifies when token counts truly
  changed.
* `rollup_and_prune` notifies once per pruning cycle (at most once per
  app start) so the dashboard reflects the new aggregate state.

Co-authored-by: in30mn1a <in30mn1a@users.noreply.github.com>
2026-05-29 22:47:51 +08:00
LaoYueHanNi bbce75fcda fix(session): 修复session log模式下子Agent token统计遗漏 (#2821)
* fix(session): 修复session log模式下子Agent token统计遗漏

collect_jsonl_files() 只扫描了两层目录,遗漏了子Agent的JSONL日志文件,
导致子Agent的独立token使用数据完全未统计到session费用中。
(仅影响session log模式,proxy代理模式不受影响)

* refactor(session): optimize collect_jsonl_files logic

- Replace two independent if statements with if-else for mutually exclusive conditions
- Remove unnecessary clone() when pushing file paths
- Add clarifying comments for main session vs subagent files
- Apply cargo fmt for consistent formatting

Performance improvement: Eliminates redundant clone() operations when
processing .jsonl files, as a path cannot be both a file and a directory.

---------

Co-authored-by: Jason <farion1231@gmail.com>
2026-05-18 10:00:57 +08:00
Jason 402570ce31 fix(usage): pricing routing, SSE lifecycle, and validation hardening
* model pricing routing: extend prefix-match families (gpt-/o1-o5/
  gemini-/deepseek-/qwen-/glm-/kimi-/minimax-) with per-family dash
  thresholds so short base IDs like gpt-5 no longer mis-match
  gpt-5-mini; strip ISO and 8-digit date suffixes via UTF-8-safe
  byte matching so claude-haiku-4-5-20251001 falls back to
  claude-haiku-4-5 pricing
* SSE collector: SseUsageFinishGuard (RAII) guarantees finish() on
  early return or panic; AtomicBool fast path lets push() skip the
  Mutex once first-event time is recorded
* validation: shared validate_cost_multiplier / validate_pricing_source
  helpers across DAO and service layers; PRICING_SOURCE_RESPONSE /
  PRICING_SOURCE_REQUEST constants replace string literals; price
  fields in update_model_pricing now reject empty / non-decimal /
  negative input before INSERT
* backfill: add backfill_missing_usage_costs_for_model so a single
  price edit only scans matching rows instead of the full log table;
  startup backfill remains full-scan
* session_usage{,_codex,_gemini}: share find_model_pricing helper from
  usage_stats; metadata_modified_nanos centralizes mtime precision
* frontend: NON_NEGATIVE_DECIMAL_REGEX + isNonNegativeDecimalString
  replace three copies of the same multiplier regex; isUnpricedUsage
  surfaces zero-cost rows that have usage tokens (cached per row to
  avoid double evaluation); invalidate usageKeys.all on pricing mutate
  so backfilled rows refresh
2026-05-14 11:53:51 +08:00
Jason f061b777b7 feat(usage): add Hermes Agent tracking + fix zero-cost bug + perf
Hermes:
- Parse ~/.hermes/state.db sessions (incl. profiles/*/state.db) into
  proxy_request_logs with data_source='hermes_session', WAL-aware
  incremental sync, Hermes-reported cost preferred over model_pricing
  fallback

Zero-cost bug (dashboard showed \$0 totals):
- GPT-5.5 family default pricing (~83% of affected rows used GPT-5.5)
- find_model_pricing_row: ASCII-lowercase normalization so
  "OpenAI/GPT-5.5@HIGH" matches seeded "gpt-5.5"
- Startup cost backfill in async task: scan rows where total_cost <= 0
  but tokens > 0, recompute via model_pricing in a single transaction

Performance:
- Add (app_type, created_at DESC) covering index for dashboard range
  queries
- Add expression index on COALESCE(data_source, 'proxy') so dedup EXISTS
  subqueries use index lookup instead of full scan; drop superseded
  idx_request_logs_dedup_lookup

Refactor:
- row_to_request_log_detail helper (3-way de-dup; fixes cost_multiplier
  \"1\" vs \"1.0\" drift between callers)
- Promote get_sync_state/update_sync_state to shared session_usage
  module (4 copies -> 1)
- run_step helper in lib.rs replaces 9 if-let-Err blocks
- maybe_backfill_log_costs returns bool to skip duplicate total_cost
  parsing in caller
2026-04-29 22:41:46 +08:00
Jason 2ee7cb4101 fix(usage): prevent double-counting between proxy and session-log sources
Proxy writes and session-log sync wrote to proxy_request_logs with
mismatched request_ids: only Claude on a native Anthropic backend used the
shared `session:{message_id}` key. Codex/Gemini and Claude-through-OpenAI
providers always produced distinct ids, so primary-key dedup never fired
and every real request was recorded twice.

Adds a 7-dim fingerprint dedup (app_type, 4 token counts, 2xx status,
model with case-insensitive match, ±10min window) wired into three layers:

- Write path: should_skip_session_insert() blocks duplicate session rows
  before INSERT, unifying the previously-divergent Claude/Codex/Gemini
  paths through a single DedupKey-based helper.
- Read path: effective_usage_log_filter() excludes already-covered session
  rows from every aggregation query.
- Rollup path: same filter applied so usage_daily_rollups never absorbs
  duplicates.

Also adds a covering index (idx_request_logs_dedup_lookup) so the EXISTS
subquery stays index-only, and a transform.rs regression test that pins
openai_to_anthropic id preservation - the missing piece that lets
Claude+OpenAI-compatible providers reuse the session: id scheme.
2026-04-29 22:41:46 +08:00
Jason 8669b408e9 fix(usage): deduplicate proxy and session log usage records
Extract message_id from Claude API responses (msg_xxx) and use it to
generate a shared request_id format (session:{msg_xxx}) between the
proxy logger and session log sync. When session sync encounters the
same request_id via INSERT OR IGNORE, it skips the duplicate.

- Add message_id field to TokenUsage, extracted from Claude responses
- Add TokenUsage::dedup_request_id() to generate shared request IDs
- Define SESSION_REQUEST_ID_PREFIX constant to eliminate magic strings
- Change proxy logger to INSERT OR REPLACE for richer-data-wins semantics
2026-04-14 10:33:41 +08:00
Jason 697d0dd6e1 fix: resolve rustfmt formatting and clippy warnings
- Apply cargo fmt across schema.rs, session_usage*.rs, skill.rs, usage_stats.rs
- Fix clippy::for_kv_map: use messages.values() instead of (_, msg) pattern
- Suppress clippy::only_used_in_recursion for intentional recursive base path
- Fix prettier formatting in UsageScriptModal.tsx
2026-04-09 16:49:13 +08:00
Jason 154342ca00 feat: add session log usage tracking without proxy
Parse Claude Code JSONL session files (~/.claude/projects/) and Codex
SQLite database (~/.codex/state_5.sqlite) to track API usage without
requiring proxy interception. This enables usage statistics for users
who don't use the proxy feature.

Key changes:
- Add session_usage.rs: incremental JSONL parser with message.id dedup
- Add session_usage_codex.rs: import thread-level token data from Codex
- Add data_source column to proxy_request_logs (proxy/session_log/codex_db)
- Add session_log_sync table for tracking parse offsets
- Background sync every 60s + manual sync via DataSourceBar UI
- Schema migration v7→v8
- i18n support for zh/en/ja
2026-04-09 16:49:13 +08:00