# Benchmarks, evals and calibration

22 entries.

- [jev-benchmarks](https://github.com/AbdelStark/jev-benchmarks) - Probability-aware evaluation for typed decision models: calibration, selective risk, latency, reproducible.
- [jevcal](https://github.com/abhixhek/jevcal) - Stop guessing thresholds: calibrate, threshold, and drift-check against an LLM teacher.
- [jev-harness](https://github.com/AntonioCoppe/jev-harness) - Confidence gates, shadow mode, recipes, and evals; reports Claude CLI at 48.9 s against Jev at 1.3 s on the same row-filter job.
- [jev-rerank-bench](https://github.com/anessbelbati/jev-rerank-bench) - Jev against Cohere Rerank, ZeroEntropy, and a chat baseline on 14 datasets, raw responses included.
- [jev-search-rerank-eval](https://github.com/zhuyansen/jev-search-rerank-eval) - Does a Jev rerank beat embedding search? 9,831 graded pairs, with the judge-circularity bias measured.
- [jev-sec-bench](https://github.com/Gaurav-Gosain/jev-sec-bench) - Blind benchmarks for prompt injection and vulnerable-code detection.
- [jev-phishing-bench](https://github.com/anisselbd/jev-phishing-bench) - Jev against Claude Haiku on 2,000 phishing emails: accuracy, calibration, latency, cost.
- [jev-spam-eval](https://github.com/bitnovus/jev-spam-eval) - Zero-shot spam filtering with Noul questions against TF-IDF baselines.
- [jev-agent-failure-benchmark](https://github.com/TokenTrim/jev-agent-failure-benchmark) - Jev against a strong LLM on the Who and When agent-failure-attribution benchmark.
- [jev-korean-benchmark](https://github.com/mahlernim/jev-korean-benchmark) - Korean understanding and medical text, with runtime and cost evidence.
- [jev-behavior-study](https://github.com/RINNECODER/jev-behavior-study) - Controlled prompt experiments on jev-1.13.0, raw results and offline verification.
- [jev-report](https://github.com/HackSing/jev-report) - Independent Chinese research report: 52 pages, 50 reproducible tests, 143 traceable data rows.
- [jev-benchmark](https://github.com/wondertwins/jev-benchmark) - Two benchmarks, chess and predator identification, one inside Jev's lane and one outside, both with results.
- [jev-dspy-lab](https://github.com/jmanhype/jev-dspy-lab) - Reproducible calibration and selective-risk benchmarks for Jev decisions in DSPy.
- [jev-eval-agent](https://github.com/vinilana/jev-eval-agent) - Personal-assistant agent with 100 mocked tools, measuring how many steps a Jev-gated agent needs.
- [jev-synergy-screening](https://github.com/PistachioAIHQ/jev-synergy-screening) - Choice and Noul questions scored against ASReview SYNERGY gold labels for abstract screening.
- [jev-orderby-bench](https://github.com/yodablocks/jev-orderby-bench) - Measures whether ORDER BY over a Jev probability is defensible: pairwise inversion, Score ordinality against a human grade, calibration, and wording invariants under a pre-registered gate; passes on 20 Newsgroups topics, fails four of six conditions on Amazon ESCI product relevance, and shows that a 40-row batched state through a DuckDB extension fails the ranking gate one row per request passes.
- [cultivar](https://github.com/pinecone-io/cultivar) - Pinecone's skill-testing CLI, with a Jev grading backend it reports at about 30 times cheaper than the LLM grader.
- [jev-eval by 4esv](https://github.com/4esv/jev-eval) - Jev against GPT-5.6 Terra on three labeled tasks: equal on the easy ones, 6.7 points lower on 77-way routing, 5 times faster, 41 to 50 times cheaper.
- [jev-benchmark by themsquared](https://github.com/themsquared/jev-benchmark) - Tool-call risk classification with the run-to-run variance reported; every wrong answer came with hedged confidence.
- [jev-research-eval](https://github.com/jgridifier/jev-research-eval) - Reproducible harness over a pinned jev-ultrafast commit, with baseline and stress suites.
- [jev-playground by hegargarcia](https://github.com/hegargarcia/jev-playground) - Jev against other models in games with explicit states, legal actions, and a measurable outcome.

Page: https://awesomejev.vercel.app/c/benchmarks-evals-and-calibration/