Home /Benchmarks, evals and calibration /JevBench

JevBench

Benchmark for Jev-class decision models: bounded rubric in, typed answer with a probability per option out, with cascade and committee experiments reported separately.

Open on GitHub Entry 492 in the readme

From the readme

JevBench — Benchmark for AI Decision Models Jev is TypeSafe AI's decision model; JevBench also evaluates Jev-compatible systems. The canonical JevBench board is https://benchmarkheaven.com/jev-models. Current results and scoring JevBench by Benchmark Heaven is a benchmark for AI decision models, including Jev-compatible systems. It measures typed decision …

Details

Section: Benchmarks, evals and calibration
owner: fstandhartinger
stars: 330
forks: 30
pushed: 2026-10-15
Language: Python
License: MIT

What it is

Form: Dataset or benchmark
Host agent: Standalone
Audience: Researchers

Maturity

Docs

reports measured numbersmocked or unclear Jev call

Useful for

Benchmarks and evals0.96

Output verification0.94

Open-model replicas0.63

Best intent matches

benchmark or calibrate Jev0.96

judge or verify an agent's output0.91

compare Jev with an LLM0.90

score or rank candidates in a pipeline0.88

run Jev on open models without the vendor0.77

+ Submit a project ★ Star on GitHub