feat(usage): support importing model pricing from models.dev (#4079)

* feat(usage): support importing model pricing from models.dev

Add an "Import from models.dev" button to the Add Pricing panel that
fetches https://models.dev/api.json, lists priced models sorted by
release date (newest 50 by default, full-text search across ~4800),
and bulk-imports the selected entries through the same
update_model_pricing command as manual entry.

- Normalize imported model IDs to match the backend's
  clean_model_id_for_pricing rules (strip vendor prefix, lowercase,
  truncate ':' suffix, map '@' to '-', drop the [1m] marker) so the
  stored rows actually match cost-attribution lookups
- Dedupe selections that collapse to the same model_id and report
  skipped duplicates in the success toast
- Invalidate usage queries on settled (not just success) so partial
  import failures still refresh the pricing list
- Keep ESC inside the picker's search input from closing the dialog
  and discarding the selection
- Add i18n keys for zh/en/zh-TW/ja and unit tests for the
  normalization, price formatting and flattening logic

Fixes #4017

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(usage): match scoped cost backfill against raw model aliases

The scoped backfill selected zero-cost rows via exact SQL string match,
but log columns store raw model strings (route prefixes, :free variants,
date suffixes), so alias rows were skipped until the next full backfill
on startup. Filter rows in Rust with the same model_pricing_candidates
normalization used by the pricing lookup; pricing decision logic is
untouched. Pre-existing gap from schema v11, surfaced by bulk import.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(usage): restrict models.dev pricing import to a single model

Each update_model_pricing call triggers a backfill pass that loads
every zero-cost usage row before filtering by model, so bulk-importing
N entries scaled as selectedModels x allZeroCostLogs full scans.
Re-importing pricing is rare, so drop the batch path instead of
optimizing it: the picker is now single-select, one import runs exactly
one update_model_pricing call and one backfill pass. This also removes
the normalized-ID dedup logic and the useImportModelPricing hook in
favor of the existing useUpdateModelPricing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Xinyu Kuo
2026-06-16 22:01:56 +08:00
committed by GitHub
parent 81d6002ace
commit 0bb3b7515a
8 changed files with 778 additions and 16 deletions
+14
View File
@@ -1509,6 +1509,20 @@
"editPricing": "Edit Pricing",
"pricingAdded": "Pricing added",
"pricingUpdated": "Pricing updated",
"importFromModelsDev": "Import from models.dev",
"modelsDevHint": "Skip manual entry — pick model pricing from models.dev",
"modelsDevPickerTitle": "Import Pricing from models.dev",
"modelsDevPickerDesc": "Select a model to import (prices in USD per million tokens). One model per import.",
"modelsDevSearchPlaceholder": "Search models or providers (full search)...",
"modelsDevAllProviders": "All providers",
"modelsDevLoadError": "Failed to load models.dev data",
"modelsDevRetry": "Retry",
"modelsDevImportButton": "Import",
"modelsDevImporting": "Importing...",
"modelsDevImported": "Imported pricing for {{name}}",
"modelsDevNoResults": "No matching models",
"modelsDevTruncated": "Showing first {{shown}} of {{total}} results — refine your search",
"modelsDevDefaultHint": "Showing the {{shown}} most recently released models (of {{total}}) — type to search all",
"cacheReadCostPerMillion": "Cache Read Cost (per million tokens, USD)",
"cacheCreationCostPerMillion": "Cache Write Cost (per million tokens, USD)"
},
+14
View File
@@ -1509,6 +1509,20 @@
"editPricing": "価格設定を編集",
"pricingAdded": "価格設定が追加されました",
"pricingUpdated": "価格設定が更新されました",
"importFromModelsDev": "models.dev からインポート",
"modelsDevHint": "手動入力の代わりに、models.dev からモデル価格を選択できます",
"modelsDevPickerTitle": "models.dev から価格をインポート",
"modelsDevPickerDesc": "インポートするモデルを選択してください(価格単位:USD / 100万トークン)。1回につき1モデルです。",
"modelsDevSearchPlaceholder": "モデルまたはプロバイダーを検索(全件検索)...",
"modelsDevAllProviders": "すべてのプロバイダー",
"modelsDevLoadError": "models.dev データの読み込みに失敗しました",
"modelsDevRetry": "再試行",
"modelsDevImportButton": "インポート",
"modelsDevImporting": "インポート中...",
"modelsDevImported": "{{name}} の価格をインポートしました",
"modelsDevNoResults": "一致するモデルがありません",
"modelsDevTruncated": "{{total}} 件中、先頭 {{shown}} 件のみ表示しています。検索条件を絞り込んでください",
"modelsDevDefaultHint": "最新リリース順に {{shown}} 件を表示しています(全 {{total}} 件)。キーワード入力で全件検索できます",
"cacheReadCostPerMillion": "キャッシュ読み取りコスト(100万トークンあたり、USD)",
"cacheCreationCostPerMillion": "キャッシュ書き込みコスト(100万トークンあたり、USD)"
},
+14
View File
@@ -1481,6 +1481,20 @@
"editPricing": "編輯定價",
"pricingAdded": "定價已新增",
"pricingUpdated": "定價已更新",
"importFromModelsDev": "從 models.dev 匯入",
"modelsDevHint": "無需手動填寫,可從 models.dev 選擇模型定價",
"modelsDevPickerTitle": "從 models.dev 匯入定價",
"modelsDevPickerDesc": "選擇要匯入的模型(價格單位:USD / 百萬 tokens),每次匯入一個",
"modelsDevSearchPlaceholder": "搜尋模型或供應商(全量搜尋)...",
"modelsDevAllProviders": "全部供應商",
"modelsDevLoadError": "載入 models.dev 資料失敗",
"modelsDevRetry": "重試",
"modelsDevImportButton": "匯入",
"modelsDevImporting": "匯入中...",
"modelsDevImported": "已匯入 {{name}} 的定價",
"modelsDevNoResults": "沒有符合的模型",
"modelsDevTruncated": "僅顯示前 {{shown}} 條,共 {{total}} 條結果,請縮小搜尋範圍",
"modelsDevDefaultHint": "預設展示最新發布的 {{shown}} 個模型(共 {{total}} 個),輸入關鍵字可全量搜尋",
"cacheReadCostPerMillion": "快取讀取成本 (每百萬 tokens, USD)",
"cacheCreationCostPerMillion": "快取寫入成本 (每百萬 tokens, USD)"
},
+14
View File
@@ -1509,6 +1509,20 @@
"editPricing": "编辑定价",
"pricingAdded": "定价已添加",
"pricingUpdated": "定价已更新",
"importFromModelsDev": "从 models.dev 导入",
"modelsDevHint": "无需手动填写,可从 models.dev 选择模型定价",
"modelsDevPickerTitle": "从 models.dev 导入定价",
"modelsDevPickerDesc": "选择要导入的模型(价格单位:USD / 百万 tokens),每次导入一个",
"modelsDevSearchPlaceholder": "搜索模型或供应商(全量搜索)...",
"modelsDevAllProviders": "全部供应商",
"modelsDevLoadError": "加载 models.dev 数据失败",
"modelsDevRetry": "重试",
"modelsDevImportButton": "导入",
"modelsDevImporting": "导入中...",
"modelsDevImported": "已导入 {{name}} 的定价",
"modelsDevNoResults": "没有匹配的模型",
"modelsDevTruncated": "仅显示前 {{shown}} 条,共 {{total}} 条结果,请缩小搜索范围",
"modelsDevDefaultHint": "默认展示最新发布的 {{shown}} 个模型(共 {{total}} 个),输入关键字可全量搜索",
"cacheReadCostPerMillion": "缓存读取成本 (每百万 tokens, USD)",
"cacheCreationCostPerMillion": "缓存写入成本 (每百万 tokens, USD)"
},