Benchmark Lab
Compare systems by metric with mandatory source-kind labels. Vendor claims, independent publications and Workstation methodology demos are never mixed silently.
Rows tagged vendor_reported quote TypeSafe (or partners) with links. Rows tagged independent quote third-party publications. Rows tagged workstation_methodology illustrate harness wiring and must not be cited as Workstation-measured production results until a suite is promoted to workstation_measured in a later lab version.
Public triage/intent-style measurements from OpenRouter and AyAutomate, plus TypeSafe’s qualitative positioning. Not a single merged leaderboard.
| System | Source kind | Source | Accuracy | p50 latency | Cost note |
|---|---|---|---|---|---|
| Jev 1.13 (OpenRouter triage) | independent | OpenRouter tutorial tables | 59/60 intent (98.3%) | 194 ms p50 | $0.0248 / 1k (author table) |
| GPT Luna (OpenRouter triage) | independent | OpenRouter tutorial tables | 59/60 intent (98.3%) | 1.106 s p50 | $0.0921 / 1k (author table) |
| Claude Opus (OpenRouter triage) | independent | OpenRouter tutorial tables | 60/60 intent (100%) | 1.957 s p50 | $2.88 / 1k (author table) |
| TypeSafe workflow Pareto claim | vendor_reported | TypeSafe announcement | Workflow-eval Pareto (qualitative) | Vendor: ~70–500 ms class | Homepage cites up to 444.6× cheaper vs refs — vendor caveat: likely high-end |
These rows do not prove Workstation reproduced the runs. Different datasets and cardinalities are not comparable as one ranking.