Northwind AI — optimization workspace
Every workload kept on its best proven configuration. $39.23 spend over the last 30 days across 6 agents.
Agent P&L — cost per attributed outcome
$0.12 / outcome · 328 attributed outcomes in 30 days
Synthetic demo datacost from: sdk 100%
attributed outcomes from: sdk_asserted 77% · auto_inferred 20% · review 2%
Evidence — what we proved
Hold on claude-sonnet-4-6 — challenger gpt-4o-mini dropped quality below the floor. Kept the loss on the board.
- savings
- 52%
- Δ quality
- -3.1%
- pairs
- 38
- non-inferior
- no
Recommend switch gpt-4o → gpt-4.1 — proven non-inferior and cheaper.
- savings
- 20%
- Δ quality
- +4.7%
- pairs
- 42
- non-inferior
- yes
Recommend switch claude-sonnet-4-6 → claude-haiku-4-5 — proven non-inferior and cheaper.
- savings
- 44%
- Δ quality
- +0.5%
- pairs
- 36
- non-inferior
- yes
Recommend switch claude-sonnet-4-6 → claude-haiku-4-5 — proven non-inferior and cheaper.
- savings
- 41%
- Δ quality
- +0.6%
- pairs
- 40
- non-inferior
- yes
Recommend switch gpt-4o → claude-haiku-4-5 — proven non-inferior and cheaper.
- savings
- 38%
- Δ quality
- +0.4%
- pairs
- 44
- non-inferior
- yes
Assurance — what we applied, and what we caught
- Invoice Field Extractor promoted (auto-graduated)holdout 0.2→0.05
- Contract Clause Classifier promoted, then auto-rolled-back (quality regression detected post-graduation (synthetic demo))holdout 0.2→0.2
- Meeting Notes Summarizer promoted (auto-graduated)holdout 0.2→0.05
⚠ 💰 Up to $32.33/mo in estimated savings — We found 32 optimizations that could save you up to $32.33/month (modeled estimate, bounded by your measured spend — not a measured result). Review your suggestions to start saving.×6
⚠ Usage spike: 249 calls in the last hour — 1109% of 24h hourly average
⚠ Hourly cost ($1.1950) exceeded 200% of 24h average ($0.1039/hr)
⚠ 10 new alerts in the last 24 hours: 5 cost spike alerts, 5 usage spike alerts. Open the dashboard to see them.
⚠ Usage spike: 254 calls in the last hour — 1114% of 24h hourly average×3
⚠ Hourly cost ($1.2401) exceeded 200% of 24h average ($0.1062/hr)
Live routing policies
| agent | mode | prod → candidate | holdout |
|---|---|---|---|
| Support Ticket Triage | shadow_sample | claude-sonnet-4-6 → gpt-4o-mini | 20% |
| Priority Ticket Router | cascade | claude-sonnet-4-6 → claude-haiku-4-5 | 20% |
| Invoice Field Extractor | cascade | claude-sonnet-4-6 → claude-haiku-4-5 | 5% |
| Contract Clause Classifier | cascade | gpt-4o → claude-haiku-4-5 | 20% |
| Meeting Notes Summarizer | cascade | gpt-4o → gpt-4.1 | 5% |
Agents
- Support Ticket Triage$11.07
- Invoice Field Extractor$9.40
- Meeting Notes Summarizer$7.69
- Contract Clause Classifier$5.58
- Priority Ticket Router$3.00
- Inbound Email Router$2.49