OpenJev Runs Agent Decisions 5x Faster Using Qwen3.5-4B
An open-source project called OpenJev accelerates AI agent decision-making by fivefold, bypassing slow token generation to read option probabilities directly from model logits.
Developer Theo Lee has released OpenJev, an open-source reproduction of TypeSafe's proprietary Jev decision service. Published under the MIT license, the project allows developers to run fast agent decision-making tasks on a single Nvidia RTX 3090 graphics card. Instead of forcing a language model to generate text or structured JSON that must then be parsed, OpenJev evaluates options directly by reading the logits from a single forward pass of a small model.
In benchmark tests comparing the system to standard autoregressive JSON generation, OpenJev achieved a 5.21-fold speedup when processing 21 binary decisions. This performance boost is achieved because the system completely bypasses token sampling and output repair. Furthermore, when handling a massive workload of 777 decisions, the system leverages parallel prefix reuse to process 20 decisions per second.
The project utilizes the open-source Qwen3.5-4B model to evaluate unstructured state data against runtime-defined questions and typed options. In terms of accuracy, the Qwen3.5-4B model running on OpenJev achieved a 0.845 agreement rate on an aligned 102-row subset, closely matching the 0.883 agreement rate demonstrated by the closed Jev service.
For AI practitioners, this approach solves a common bottleneck in agent architectures, where simple routing, call retries, or claim verifications are slowed down by text generation. By returning raw probability scores directly, OpenJev eliminates the latency and parsing errors associated with structured outputs. The repository is hosted under the name TheoLeeCJ/SemIf, and a web-based demonstration utilizing browser WebGPU is available at openjev.com.
This is our own summary of reporting by AlphaSignal
