Home /Open models and replicas /openjev-sglang
openjev-sglang
Jev-compatible API endpoint on SGLang, prefill only.
From the readme
openjev-sglang A server implementing the TypeSafe/Jev HTTP API with Qwen3.6-35B-A3B on SGLang. Each container one B200 with SGLang 0.5.19's Rust frontend, radix caching, and breakable prefill CUDA graphs. A separate Python API process uses FastAPI, uvloop, the Rust-backed HF tokenizer, and pooled asynchronous HTTP connections to SGLang on localhost. CUDA …
Details
- Section
- Open models and replicas
- owner
- ekzhang
- stars
- 207
- forks
- 22
- pushed
- 2026-09-18
- Language
- Python
What it is
- Form
- Proxy or service
- Host agent
- Standalone
- Audience
- Agent builders
Maturity
Docs