typesafe-sdk-js
TypeScript and JavaScript client with answer types inferred from your questions.
Home / All projects
Search and filter every project on the list. Local search finds candidates; Jev reorders the top 30.
TypeScript and JavaScript client with answer types inferred from your questions.
Same `TypeSafeClient` interface backed by an LLM API, so you can compare Jev against a chat model on identical questions.
Agent skills for designing questions, building workflows, and evaluating them.
How to install the official skill in Claude Code, Cursor, and friends.
`@ai-sdk/typesafe-ai` exposes Jev through `experimental_evaluate`.
Vercel's agent framework; Jev is the typed judge in its evaluate step.
The Vercel AI SDK in your terminal, with an evaluate path that runs on Jev.
Replaces the compaction summary with Jev decisions: every tool call and result scored in one request, stale ones dropped, everything kept stays verbatim.
Routes each task to the cheapest Claude model that can handle it.
Judges every tool result before it enters context, so the window fills slower instead of being cleaned later.
Context-pruning proxy for Claude Code and Codex, with the savings measured rather than claimed.
Rust CLI and hooks that rank installed skills for the next step using live session context, with abstention.
Local proxy that picks model and effort per message and leaves the cached main chat alone.
Per-prompt router over skills, MCP servers, agents, and commands, and it measures whether the injection helped.
A Stop hook that keeps the agent from stopping too early, judged against plain-language rules.
MCP server, embeddable decision model, and an escalate-only plugin that can make the harness stricter but never looser.
Command-line toolkit that coding agents hand judgment-heavy work to, one typed Jev workflow per request.
Agent toolkit whose `/d` command picks the specialist agent, skill, and pipeline with one Jev call, plus an optional Jev auto-compact plugin.
Compaction that asks Jev which tool calls still matter and keeps the rest verbatim, with adapters for Claude Code, Codex, OpenCode, and raw API transcripts.
Trims long Bash output with Jev after the command runs and before the model sees it; short output, errors, and structured formats pass untouched.
Scores your standing rules against each prompt and delivers only the ones that apply, once per session.
Stop hook that blocks an unverified "done": reads the transcript for evidence and, only when files changed with no passing check since, spends one four-question Jev call; fails open on every error path.
Hands the Claude Code, Codex and pi steps that need no text output to Jev, with a typed escalation contract for everything it should not decide.
Hook that asks Jev which installed skill fits each prompt and logs the answer; in the author's week of shadow mode, 28 of 539 suggestions were followed by a call of that skill, and the author then removed it.
Picks model, thinking depth, and speed mode for every Codex turn.
Uses Jev to select the model and reasoning effort for Codex subagents, with confidence gates and a Sol fallback.
Has Jev pick the reasoning effort and how long to hold it for a running Codex task, on a patched Codex CLI built from upstream source.
A measured tool-call gate plus a `jev_ask` tool for typed answers inside Pi.
Guardrails that steer instead of interrupt: irreversible calls, off-task calls, stuck loops, unverified done claims, about 250 ms each.
Auto-approves bash, write, and edit calls semantically and fails closed when it cannot decide.
Semantic tool routing and typed decisions as Pi tools.
Automatic model routing for Pi through the Vercel AI Gateway.
The verbatim compaction idea, ported to Pi.
Hybrid harness for Pi: an LLM writes the code, Jev reflexes gate every call as allow, confirm, block, warn, or steer.
Jev as the Pi coding agent's quiet decision layer.
Pi extension exposing Jev judgments as five Pi tools.
Batched evaluation tool, terminal playground, and a typed API for Pi extension authors.
Checks every side-effecting tool call against what you said earlier in the session, so "review only" still holds after compaction.
Checks Pi tool calls, tool outputs, and replies for risky actions and prompt injection, with user approvals, context re-checks, secret scrubbing, and optional task pinning.
Opt-in typed evaluation and semantic search over MCP tool results, behind a per-server data-egress allowlist.
Coding agent and desktop app built on Pi that hands 38 routine decisions to Jev, from which tool output enters the context to prompt-injection screening and checking a "done", each one switchable between active, shadow and off with every verdict in a local ledger; pre-release.
Names the one skill worth loading before the model call; stdlib only, about a tenth of a cent per turn.
Confidence-aware skill routing with an abstain path.
Typed decisions, ranking, verification, and an opt-in tool gate.
Lets Hermes and similar agents ask Jev directly.
Four Hermes tools for atomic checks, routing, and rubric scoring; listed in the Hermes plugin catalog.
Approves, denies, or escalates flagged shell commands before they run; vendor-reported speedups.
Typed tools and probability cards for Agent Zero.
Self-hosted, versioned skills library served over MCP, with Jev recommending which skill to load.
The first MCP server for Jev, and still the most linked.
Classify, score, check, match, and screen, with confidence on every answer.
ACP and MCP adapter that pairs Jev with any LLM for computer use and typed decisions.
Eval-first MCP server: prototype a question, map it over many items, then measure variants against labeled examples with a threshold sweep.
Recursive language model layer for Claude Code, Codex, Gemini, and OpenCode that keeps full sources in a local evaluator; Jev is an optional leaf for reranking, verification, classification, and semantic joins, with budgeted, checkpointed batches.
Skill for writing and improving programs that call Jev.
Finds the fuzzy judgment in a system and turns it into small Choice, Score, and Noul primitives.
Sends a coding agent's closed judgments to Jev instead of the chat model.
Agent-skill directory that can score subjective evaluation criteria with Jev.
Skill for deciding where a typed judgment belongs at all and what stays in code; a companion to the official skill, not a replacement.
Hands an agent's small decisions to Jev: which model answers the turn, which skills to load, which passages matter, which turns survive compaction.
Coding CLI where Jev routes a plain-language request to one deterministic workflow, and that workflow asks Jev fixed-choice questions about the evidence it gathered.
Has an LLM write a task-specific harness that turns observations into Jev questions, then freezes it and improves it from rewards and full execution traces.
Skill that packs the Choice, Score, and Noul API shapes, confidence thresholds, and 150+ community projects with a code sketch each, so an agent can design without fetching the docs.
One request picks both the operation and the target element from an indexed DOM table; a small LLM only writes typed text. Zürich to London booked in 7.1 seconds.
OCR the screen, classify the next action, click; about $0.0002 a step on macOS.
The same loop on a real Android phone; nine Uber actions in 21 seconds in the demo.
The first community browser agent on Jev, with a demo GIF.
Intent and target decided per spoken word in about 300 ms, often before the sentence ends.
An LLM plans, Jev decides; library, CLI, and MCP server.
One grounded Jev and Playwright core behind a typed SDK, a persistent CLI, and an MCP server.
Android sub-agent over USB running observe, normalize, decide, mutate, verify, with Jev deciding.
Browser agent that spends one Jev request per step to pick the action.
Codex skill and plugin where Jev handles navigation, clicks, and scrolling and Codex keeps typing and verification; reports browser steps 5 to 10 times faster.
Codex computer use where Jev picks the element, action, completion, and risk from on-screen text, no screenshots sent; Chinese readme.
Job-hunting skill that drives Chrome over CDP and has Jev score every link and listing.
Lets Jev choose each read-only social research step while socai runs it in a real Chrome session and streams inspectable Instagram, TikTok, or LinkedIn evidence into a report.
Voice and typed computer use for macOS: Jev picks the next on-screen action from the Accessibility tree, with no screenshots.
Android automation where one Jev request picks both the action and the target element from the accessibility tree, executed over ADB.
Runs browser tasks from a plain-language instruction over CDP or Vercel's agent-browser, with Jev selecting operations and targets and code checking confidence before it acts.
Runs a delegated GUI task for Codex through the real Chrome session or Windows UI Automation, with Jev picking the control at each step.
Semantic ifs from open models on a single 3090; the most starred independent replica, formerly openjev.
Open option scorer that reads candidate logits instead of generating JSON.
0.6B replica with parallel decisions, dynamic candidates, and an end-to-end training pipeline.
Jev-compatible API endpoint on SGLang, prefill only.
Educational visual-inference variant on Apple Silicon: shared context, direct candidate scoring.
Small open decision model on Qwen3.5: state plus typed questions to calibrated probabilities.
Parallel constrained decisions for any MLX model on Apple Silicon, one forward pass.
Preregistered experiment on a frozen Qwen3-4B: read the option letter's logits, skip the JSON.
Typed calibrated decisions in one forward pass on Gemma 4 E2B and Gemma 3 270M.
Non-autoregressive decision engine on ModernBERT with calibrated uncertainty and an in-browser WebGPU playground.
Parallel decisions for CUDA LLMs through a vLLM API, with game-agent examples and benchmarks.
A stronger one-pass scorer with a head-to-head benchmark against the jevlike starter design.
Typed, calibrated decisions from any open-weights model in one forward pass, on Hugging Face and vLLM.
Bilingual local decisions from context, questions, and candidate answers.
Study of Jev-style decisions on stock 1.5B to 8B models on a laptop, with a Hugging Face demo.
LLM gateway that mimics the TypeSafe response shape, useful as a stand-in while you wait for a key.
Hugging Face Space demonstrating RLCD-style parallel decoding on Qwen2.5-1B.
Open decision layer answering a finite option set with probabilities split into execute, confirm, and reject. Independent replica, not TypeSafe weights.
Non-autoregressive open decision model reporting under 15 ms locally, as a drop-in alternative to Jev.
Turns any off-the-shelf LLM into a Jev-style decision layer.
Typed JSON inference with DiffusionGemma, benchmarked against Jev.
LoRA adapter and readout head on Qwen2.5-0.5B that answers many typed questions in one prefill; trains in under two hours on a MacBook, held-out ECE 0.065, speaks the TypeSafe wire format.
Reads next-token logits from any Hugging Face model for choice, rubric, and support questions; public demo API with no key.
Jev-compatible decision server on DiffusionGemma 26B through vLLM, images included, hosted free on [Codiv](https://codiv.ai).
Prefills once and scores every option in one padded pass on Gemma 3 4B with MLX; plays Doom from the terminal in the demo.
Self-hosted System One API on the 400M GLiFormer, with benchmarks that say where it trails Jev.
End-to-end stack for auditable data construction, Qwen3.5-0.8B training, fixed Mind2Web and OOD evaluation, local serving, and a preliminary RLCD baseline.
Plays ten browser games from the frame alone on a fine-tuned Qwen3.5-0.8B, one forward pass per move, open weights and a browser demo.
Non-autoregressive 151M decision engine with a WebGPU browser runtime, reporting 77.1 percent top-1 and 1.44 percent calibration error on its own benchmark.
Adapts local language models to runtime-defined Choice, Score, and Noul questions and returns typed answers with probabilities.
Offline server wire-compatible with the System One endpoint, checked against the official SDK; reports 80.5 percent on JevBench's public items against 86.6 percent published for the hosted API.
Gateway that turns an existing SGLang or vLLM deployment into a Jev-compatible decision service by scoring candidate tokens, with no training and no model changes.
Local-first take on the System One idea: unstructured state in, typed probabilistic decisions out, without generating a token.
Open-weight typed-decision models on Qwen3.5 with calibrated per-option probabilities in one forward pass; reports 80.1 percent on JevBench against 75.3 for Jev 1.13.
Turns an open Hugging Face model into a calibrated decision endpoint served through vLLM, with no fine-tuning at the first two levels.
Pulls and serves open decision models such as Laya, kev, and JevK5 behind a TypeSafe-compatible local endpoint, the way Ollama serves LLMs.
Decision model on Qwen3-0.6B that answers without decoding tokens, reports 79.25 percent top-1 on the public Typed Decisions benchmark.
Multimodal decision model on Qwen3.5 that scores candidates against images and video as well as text, with training code and open weights.
Data recipe and training code that fine-tune Qwen3.5-4B into an open-weight decision model on Together AI.
Qwen3.5 replica with distilled LoRA weights, ranked second of 76 systems and first among open ones on JevBench v1.4.
Serves `/v1/systemone` on DiffusionGemma by reading typed answers and text spans off a seeded, pinned canvas, with no generation.
Open multimodal System One model, 0.8B to 27B, for screenshots, photos, video and text.
Open-weight ModernBERT-base fine-tune that answers Noul, Choice, Score, and multi-label questions with a probability per option, serves `/v1/systemone` for the official Python SDK, and runs in the browser through ONNX; independent replica, not TypeSafe weights.
Runs several coding agents locally with Jev as judge and orchestrator.
Staged code-review workflow with a local dashboard.
Local-first MCP plugin for continuous quality review by coding agents.
Supervises a software factory of agents, with Jev making the go and no-go calls.
Every file in a PR judged against Clean Code rules, then reviewed by an LLM.
Classifies commit diffs and messages: bug fixes, security fixes with CWEs, change types.
GitHub Action for submission review and PR classification with Jev, no text-generation model in the loop.
Pulls large repositories and triages their issues with typed Jev questions.
Semantic linting: rules in plain language, each file judged by Jev, run locally or in CI.
Read-only Go CLI that checks files against coded rules such as hidden side effects and weak error handling.
Turns the preferences in your AGENTS.md into a linter that runs on code changes and reports back to the agent.
Pre-commit hook: one Jev call judges whether the commit message matches the staged diff, plus debug leftovers, unmentioned work, and a credential belt; warns except on a secret, which it blocks.
Grades text and markdown files for AI slop, grammar, and technical documentation quality, and guides an AI agent to auto-fix violations.
GitHub App that asks Jev whether a pull request is safe to approve or needs a specialist, then maps the verdict to a check run.
C/C++ compiler driver that asks Jev whether to inline each discretionary call site, from the LLVM IR and the original source.
Checks the code against the requirements in a Markdown spec on every commit and fails the build when the two drift apart.
Pytest plugin that asks Jev whether plain-English claims about a test's text hold, all in one request, and fails the test with each claim's probability unless Jev is at least 80 percent sure.
`--diff` gates a PR in CI on a rule written in English, `--tests` lists the test files a diff can affect, and plain `jgrep` greps code by what it does; one Noul per chunk, 16 chunks per request.
Shifts every LLM call to the cheapest model that can handle it, policy in YAML, decision in about 180 ms.
Picks Cursor, Claude Code, Codex, or OpenCode plus model and effort for a task, then launches it.
Measures on your data when Jev beats other models, then routes accordingly.
Models, subagents, skills, MCP tools, and CLIs as one candidate set; Jev picks, the router enforces permissions and risk; reports 44 percent first-five tool-call hits against 24 for DeepSeek on Toolathlon.
Local gateway for Codex and Claude Code that sends the "which tool next" decision to Jev and everything else to your usual model.
Jev picks the worker tier for each step in OMP and Pi, keeping planning and review on a strong model and bounded work on cheaper ones.
Local OpenAI, Anthropic, and Responses-compatible proxy that serves one Jev call per turn to answer both the model route and the thinking level for its virtual model jevonian/auto, with code filtering candidates by protocol, context window, effort floor, and spent quota windows first, and pinned models or explicit routes skipping Jev entirely.
TypeScript AI SDK where decide, via Jev, is a peer of generate and stream: one typed-judgment call routes model choice, prunes context, and picks MCP tools.
Source selection, query understanding, and relevance ranking for web search.
Jev as a calibrated reranker: one call, up to 30 documents, a probability per document.
LlamaIndex reranker and router, cheaper than an LLM judge.
Recursive choice over a taxonomy, past the 255-option cap.
Classifies OCR'd legal agreements through the FOLIO Document Types ontology with recursive Jev Choices, beam search, confidence-gated leaf stopping, and context-length benchmarking.
MCP tool that scores a batch of files, URLs, or snippets for relevance so the agent opens only what matters.
Rust CLI and MCP server that finds real, maintained repos and crates for a plain-language request, with Jev scoring the candidates.
Greps by meaning instead of by regex: every line gets a probability from Jev, and meanings combine with AND, OR, and NOT.
A shadcn/ui registry block: keyword hits on the first keystroke, re-ranked by Jev a moment later, and keyword order stands if the call fails.
Browser extension that searches the page you are reading by meaning, in a Ctrl+F style box, with your own key and no backend.
Nine runnable notebooks combining Gemini embeddings and Milvus retrieval with Jev decisions for reranking, filtering, search stopping, routing, cache reuse, curation, guardrails, and evaluation.
Searches a PDF by meaning in the browser: pdf.js extracts the lines locally, Jev answers one Noul per line in batches of 16, and matching lines light up page by page, ranked by probability.
Searches local documents with BM25 or hybrid retrieval, reranks candidate passages with batched Jev Noul judgments, and measures seven retrieval pipelines on BEIR NFCorpus.
PostgreSQL extension that answers plain-language questions about your tables.
DuckDB worker that exposes choice, noul, and score as lateral-joinable table functions in SQL.
SQL with natural-language predicates over SQLite: filter, rank, and classify rows by meaning, batched and cost-guarded.
Adds Jev Noul, Choice, and Score judgments to SQLite through a loadable C extension and Python wrapper, with scalar functions and batched virtual-table queries.
Scores OpenTelemetry log signal before paying for LLM analysis.
Sifts Parquet and JSONL training data at more than 1,500 rows a second.
Home Assistant integration: ask a question about your house, get a probability, choice, or score as an entity.
Eight small examples from mechanical and electrical engineering: CAD routing, FEM triage, DFM screening, BOM alignment.
Links records across two datasets from a match rule written in plain English, from Python, the shell, Stata, or R, and reports F1 0.73 against 0.69 for tuned string matching on NBER patent assignees to Compustat.
Pre-alpha PostgreSQL extension for categorical classification with Jev.
Ticket triage and file tagging on a typed-judgment model; reports 78 percent fewer tokens and 96.1 percent accuracy against a 93.7 percent baseline.
Asks a clinical trial report for systematic-review data by voice, text, or a questions file; every answer is a verbatim quote with its file and place.
One request per page picks among 261 IRS forms and seven page kinds; reports 100 percent on its corpus at $0.001 a page, 34 times cheaper than the LLM pipeline it replaced.
Semantic SQL for PostgreSQL, with Jev answering the predicates.
DuckDB extension that asks a question of every row and returns a real SQL type.
Gives every stored agent memory a lease and asks Jev whether new evidence ends it; [live demo](https://invalidate-playground.vercel.app).
Parses PDF, DOCX, PPTX, or text locally, then asks Jev one closed-set relation question per candidate entity pair and exports a graph with per-edge probabilities and page evidence to JSON, CSV, or Neo4j.
Packs 32 items into one request for bulk classification and calibrates the confidence cut that sends the least-sure rows to a person, reporting 533 items a second at 89.2 percent agreement with human labels.
Crawls a site, checks it against 52 SEO rules, has Jev judge every page, and writes PDF, spreadsheet, and Markdown reports.
Zero-shot named entity recognition for Chinese and English: code proposes candidate spans at punctuation and Jev choice questions nominate, verify, and fix the boundary of each entity; reports 73.7 average strict F1 on 12 NER benchmarks against 72.1 for Qwen3.8-27B.
Semantic MCP firewall that screens every tool call, result, and description; reports 94 percent block recall at about $0.00002 a check.
Auto mode for Claude Code, Codex, Cursor, Gemini CLI, Pi, and OpenCode: risk-scores each tool call as deny, ask, or allow and flags prompt injection in results.
Chat moderation with editable rules.
Does the cited paper support the sentence citing it? Claude finds the quote, Jev scores it, a human decides.
Paste text, get diagnostics, like a compiler for prose.
Prose linter for AI writing tells: countable rules plus one judgment model.
Measures how well Jev spots real credentials in file snippets, with the hard config-shaped cases scored separately.
Low-latency audio censorship proof of concept: Jev typed decisions drive ffmpeg.
Scans a codebase for hidden or data-stealing behavior before you run it; a clean report is not proof, and it says so.
AI SDK middleware and proxy that runs seven Jev checks on every LLM response before the user sees it; no accuracy numbers yet, and it says so.
Sends an agent skill's text to Jev before installation and returns a verdict with category scores, without loading the skill into the agent's context; the submitted text is not redacted for secrets.
Telegram bot that removes spam and scam messages from group chats on calibrated-confidence classification.
Checks claims against cited excerpts with Jev in an English and Romanian fact-checking site, with a public integration example and a closed-source full application.
Browser extension that removes page clutter with reusable template rules.
Chrome extension that asks "is this element an ad?" per DOM node; a toy, and it says so.
Live tone labels for Bluesky posts and drafts.
Puts a live meter on any video: every sentence scored on five questions, rendered as a 16:9 edit, a whole debate for about two cents.
Describe your startup idea; Jev says kill it, fix it, or ship it.
Voice-driven auto-advance for Slidev on Cloudflare Agents.
Asks Jev the question every ten minutes, using the actual headlines.
Turns privacy policies into nutrition-style labels with plain-language answers, Jev confidence scores, and suggested source clauses.
Menu-bar Swift app turning spoken commands into Jev typed decisions and macOS actions.
Game of Thrones roleplay where a story model writes each scene and Jev answers five typed questions that drive the header, soundtrack, art, and next prompt.
Jev experiments starting with a local CV-screening workbench, each with its own measured results.
Small harness connecting evidence, Jev judgments, permitted actions, and verified outcomes.
Neon Function proxy for the Neon AI Gateway with Jev routing in front.
Chrome extension that finds sponsor reads from the transcript or live audio and jumps past them; code owns every timestamp, under a cent an hour in transcript mode.
Rust CLI and MCP server for SEO and GEO checks over DuckDuckGo results, scored by Jev.
Neovim plugin: ask the buffer a plain-language question, Treesitter splits it into functions, Jev scores each one, and the answers land in quickfix ranked by probability.
Browser extension that reads the caption track and paints a per-segment sponsor probability on the seek bar before the intro ends, no crowd database; reports 77 percent of SponsorBlock's sponsor seconds caught over 23 videos at $0.0008 a video.
OpenPoke fork that moves email screening, a tool-call guardrail, and search reranking onto Jev, with an A/B against the Sonnet call it replaced and an adversarial run on the injection gate.
Read-only Gmail triage: three Jev questions per message sort it into one of five trays with an urgency score.
Config-driven inbox classification that tags, moves, flags, and notifies, through TypeSafe directly or OpenRouter.
Reads a chat log locally and labels every message with the three likeliest of twelve emotions and thirty-five intents.
Talk to macOS: local whisper.cpp transcribes, one fan-out Jev call picks the action and its typed arguments, and code runs it.
Read English aloud and see it scored: streaming ASR listens, Jev judges word by word, and the total is arithmetic in code.
Chat-shaped command bar where Jev picks the tool, the arguments, and the reply shape, and code builds the answer from the tools' own data, so no model writes the text.
Mines repeating motifs from a MIDI melody, scores every candidate in one batched Jev call, and highlights and plays the winner on a piano roll; on an orchestral Swan Lake the top motif is the swan theme.
Turns one text box into the matching card as you type, judged by a single parallel Jev call, with a keyword fallback when no key is set.
Plays Super Mario Bros. from structured emulator state; Jev picks the NES controller input directly.
Camera-only drone in MuJoCo with Jev in the loop at 2.5 Hz.
Plays the original StarCraft shareware through keyboard and mouse, action probabilities recorded.
Civilization II in a browser, full-game harness, live action probabilities.
Stealth game where Jev makes the guards' judgments and deterministic code owns the world.
One Choice per tick; legal moves and facts generated in code.
Browser-native Doom agent with structured spatial state and live decision telemetry.
Generation 3 Pokémon through Showdown and a real FireRed ROM.
Zero-shot English goals on a simulated Franka arm; Jev chains hardcoded primitives.
Command line for LLM-piloted robots across seven bodies, with an optional Jev stepper that picks among calls without ever writing a joint angle.
Snake controlled by parallel Jev assessments, one API call per tick.
Three.js driving simulator where Jev picks steering and speed from sampled paths up to four times a second; [drive it](https://jevpilot.standardagents.ai).
Top-down car in the browser sending four typed questions every 200 ms, with confidence-gated overrides in code.
Pokemon Red on PyBoy: code owns the route and the arithmetic, Jev picks only at branches, and every battle turn logs a faint prediction scored by Brier against what the RAM says.
Plays Clash Royale with Jev choosing cards and placements from Qwen battlefield vision and local OpenCV hand and elixir recognition.
Kills the dragon on a vanilla server with a frontier model planning and Jev selecting every player action; the logged run took 131 Jev decisions and 35 planner calls.
Browser workbench for embodied experiments in MuJoCo where every robot step is a visible Jev decision, backed by a local model or a cloud API.
Connect Four with nine swappable query strategies for the same model, every answer graded against engine ground truth in a live inspector, and a match runner that plays 100 games a minute: representation changes alone moved the win rate 52 points.
One trade decision every Monad block, on Kuru MON-USDC, about 300 ms each.
Backtests Jev as a buy, sell, or hold trader on NQ order-book data.
Runs three Jev-driven trading bots against OKX perpetuals with a live dashboard, paper trading by default and real money behind explicit flags.
Probability-aware evaluation for typed decision models: calibration, selective risk, latency, reproducible.
Stop guessing thresholds: calibrate, threshold, and drift-check against an LLM teacher.
Confidence gates, shadow mode, recipes, and evals; reports Claude CLI at 48.9 s against Jev at 1.3 s on the same row-filter job.
Jev against Cohere Rerank, ZeroEntropy, and a chat baseline on 14 datasets, raw responses included.
Does a Jev rerank beat embedding search? 9,831 graded pairs, with the judge-circularity bias measured.
Blind benchmarks for prompt injection and vulnerable-code detection.
Jev against Claude Haiku on 2,000 phishing emails: accuracy, calibration, latency, cost.
Zero-shot spam filtering with Noul questions against TF-IDF baselines.
Jev against a strong LLM on the Who and When agent-failure-attribution benchmark.
Korean understanding and medical text, with runtime and cost evidence.
Controlled prompt experiments on jev-1.13.0, raw results and offline verification.
Independent Chinese research report: 52 pages, 50 reproducible tests, 143 traceable data rows.
Two benchmarks, chess and predator identification, one inside Jev's lane and one outside, both with results.
Reproducible calibration and selective-risk benchmarks for Jev decisions in DSPy.
Personal-assistant agent with 100 mocked tools, measuring how many steps a Jev-gated agent needs.
Choice and Noul questions scored against ASReview SYNERGY gold labels for abstract screening.
Measures whether ORDER BY over a Jev probability is defensible: pairwise inversion, Score ordinality against a human grade, calibration, and wording invariants under a pre-registered gate; passes on 20 Newsgroups topics, fails four of six conditions on Amazon ESCI product relevance, and shows that a 40-row batched state through a DuckDB extension fails the ranking gate one row per request passes.
Pinecone's skill-testing CLI, with a Jev grading backend it reports at about 30 times cheaper than the LLM grader.
Jev against GPT-5.6 Terra on three labeled tasks: equal on the easy ones, 6.7 points lower on 77-way routing, 5 times faster, 41 to 50 times cheaper.
Tool-call risk classification with the run-to-run variance reported; every wrong answer came with hedged confidence.
Reproducible harness over a pinned jev-ultrafast commit, with baseline and stress suites.
Jev against other models in games with explicit states, legal actions, and a measurable outcome.
Hosted arena that puts Jev against an opponent judge you connect and takes your vote before revealing which was which, with latency, cost provenance, and self-reported confidence; a vote records preference, not verified correctness.
Benchmark for Jev-class decision models: bounded rubric in, typed answer with a probability per option out, with cascade and committee experiments reported separately.
Finds the examples a Jev function is least sure about, asks you to label them, and improves the function with GEPA.
Measures what a Jev request is billed and how its answers hold up under batching, translation, and rewording, from 3,455 billed requests with public raw logs.
Runs a trace's agent, quality, and security evals as one batch of typed Jev questions instead of separate LLM-judge calls.
110 use cases, games, and model challenges with editable prompts and A/B comparisons; a community org, not the vendor, formerly under BunsDev.
From support routing to a 3D driving simulation with visible sensor inputs.
Nader Dabit's grab bag of small Jev experiments.
Sixteen typed judgments update as you type, on Val Town.
Ask a question, get yes, no, or maybe, with web search when needed; no signup.
Rotation and column chosen from holes, stack height, and bumpiness.
Test a post against 10,000 synthetic personas before you publish it.
Ask a question, one choice over twenty answers picks the reply and shows the click-to-answer latency; [live demo](https://willprout.github.io/magic-8-ball/).
Rust CLI playground for experiments around Jev.
Can a System One model steer a music composition through typed classify, score, and pick decisions alone.
LangGraph demo that sends a mocked email to Jev and routes on the typed Choice it returns.
Hosted demo filing posts by type, quality, sentiment, and tone.
Hosted robot interface answering nine typed questions per turn to decide act, ask, or shrug, no generated text.
Terminal interface where OpenAI answers and Jev separately scores relevance, reliability, and quality.
Turns a SaaS landing page into Markdown and asks Jev ten bounded Choice questions about what a first-time visitor understands, shown as a founder teardown.
Reads the most-discussed Hacker News threads comment by comment with Jev and reduces each one to a single verdict, served live from Postgres.
Shared 1000 × 1000 emoji canvas where Jev joins each stroke with one Choice over named emoji and square pairs, its pick sampled from the probabilities; [paint on it](https://chriswijnia.com/lab/emoji).
Shell verbs for agents and humans: pick, rate, check, rank, triage, guard.
Typed semantic decisions for Unix pipelines and CI.
Ask a yes/no question of every function in a codebase; grep whose pattern is a question.
Noul, choice, and score answers as numbers from the shell.
Prints the lines that fit a plain-English description, streaming from `tail -f` under a spend cap, and reports F1 0.91 on SMS spam against 0.72 for a keyword grep.
Typed questions in, structured JSON answers out.
Dependency-free Python CLI wrapping Choice, Score, and Noul.
npm CLI with the key in the OS keychain; typed judgments from the shell.
Sorts PDFs and text files into category folders, one Jev Choice per document, with a local Laya backend as the alternative.
Pipes and composes Jev judgments over JSON and NDJSON with map, reduce, rank, and rate, then applies policy through an explicit offline gate.
Filters log lines and other text streams by meaning, one Noul per line in micro-batches, streaming from `tail -f` with grep's flags and exit codes.
Finds the files, declarations, and verbatim excerpts a coding agent needs by asking Jev which declarations answer a plain-language question about the repo.
Searches the web for an agent and has Jev score and dedupe results and page chunks, so only the relevant text reaches the model.
Go client that returns typed judgments and probabilities.
Reranker package whose Jev adapter sends every candidate document as a Noul question in one request, sorts by the returned probabilities, and keeps original indices; shipped in the v0.3.4 release.
Rust client with async and blocking backends and observable retries.
Elixir client built for OTP: reply to Jev from a GenServer and pattern match on the answer.
TypeSafe as a structured-output provider for RubyLLM 2.
Laravel integration with typed responses, async requests, and testing fakes.
Haskell DSL with typed packets and inferred answer types.
Small TypeScript client for asking questions about your own data.
Zod-style schemas over Jev: validate the state locally, then project typed answers.
n8n community node for yes/no, choice, and score questions.
Async Python client, probabilities and choices out, no prose to parse.
Turns Rust enums and structs into Choice, Score, and Noul questions with compile-time-checked, confidence-gated answers.
Rails integration built on the community Ruby gem.
Elixir port of the TypeScript AI SDK with a TypeSafe provider.
Ruby client for decision models with OpenRouter and TypeSafe providers behind one interface, stdlib only.
Zod 4 schemas with semantic rules: shape checks stay in Zod, meaning checks go to Jev in one request and come back as Zod issues.
Community .NET SDK with typed Noul, Choice, and Score questions.
Swift 6.4 SDK following the Python SDK's API, on Apple platforms and Linux.
PHP client with sync calls, Guzzle promises, and PSR-3 logging.
.NET 10 SDK for Jev decisions through TypeSafe, OpenRouter, Vercel AI Gateway, and compatible endpoints.
Turns Choice, Score, and Noul questions into Python functions over lists and DataFrames, with deduplication, caching, escalation of unsure rows to an LLM held to the same labels, and a TypeScript port on npm as hunch-jev.