Home /Benchmarks, evals and calibration /jev-eval by 4esv
jev-eval by 4esv
Jev against GPT-5.6 Terra on three labeled tasks: equal on the easy ones, 6.7 points lower on 77-way routing, 5 times faster, 41 to 50 times cheaper.
From the readme
jev-eval Benchmark TypeSafe Jev against any OpenRouter model or a local checkpoint on labelled classification data: accuracy, calibration, confidence distribution, latency, cost. Ships five tasks; the results below are Jev, GPT-5.6 Terra, and three open Jev-shaped models. Results 300 items per task. jev-1.13.0; openai/gpt-5.6-terra via OpenRouter; Laya …
Details
- Section
- Benchmarks, evals and calibration
- owner
- 4esv
- stars
- 1
- forks
- 0
- pushed
- 2026-10-09
- Language
- Python
What it is
- Form
- Dataset or benchmark
- Host agent
- Standalone
- Audience
- Researchers
Maturity
Docs
reports measured numbers