Overview & Result
you can make any open source model behave like jev with just a bit of inference engineering. it's shockingly easy. to prove it, we built a new endpoint we're calling deepseek-v4.1-flash-jev. see the demo below. here's how it's done: sglang (an inference engine) offers a scoring endpoint in addition to the normal generation one. in scoring mode, given an input & set of possible answers, it forces the model to produce probabilities for each one. example: > input: what is most common letter in abcccde? > possible answers: a, b, c > output: (c, 0.9), (b, 0.0.5), (a, 0.05) getting the above behavior instead of streamed output is as simple as using sglang's /v1/score endpoint instead of /generate. there's just one other trick required. for deepseek, you have to add a closing think tag before the response. this forces a direct answer instead of a reasoning trace. if you want reasoning, you can do that too, but imo that makes things too slow to be worth it. dsv4.1 flash is not as good as jev, but if we had enough spare compute to experiment with this same approach for a larger model then i think the decision quality would be at least as good, if not better. also, somewhat unrelated, i think decision-making models kill all prospecting & sourcing work. i would have absolutely killed to have jev or similar when i was recruiting @mintlify. absolutely incredible.
How Jev fits in the loop
- Ingest real-time application state and relevant contextual parameters
- Format the decision problem as a bounded Choice or Noul schema
- Query Jev to receive a typed probability distribution in sub-50ms
- Execute downstream actions or route tasks according to the winning choice
How to reproduce
- Inspect the original showcase and source material at https://x.com/skeptrune/status/2101209390992994570
- Verify the bounded prompt and input candidate schema configured for Jev
- Benchmark decision latency and classification accuracy against baseline models
Why this build matters
Demonstrates a real-world, cost-effective implementation of Jev in a specialized agent workflow scenario, replacing expensive generative calls with fast typed decisions.
Reported performance
Reported by authorLatency: Sub-50ms deterministic decision window
Limitations
- Task accuracy is bounded by the precision of the defined candidate choices
- Third-party external dependencies and network latency may affect total workflow duration