DEMO / RECORDING
Overview & Result
Benchmark evaluation testing Jev paired with Mercury 2.5 against GPT-6 Astra. Jev picked the required tool from WebMCP, while the fast LLM generated arguments, achieving 100% completion at a fraction of standard cost.
How Jev fits in the loop
- Extract WebMCP available tools on benchmark step.
- Jev Choice identifies correct tool function in tens of milliseconds.
- Fast small LLM populates parameters and verifies result.
How to reproduce
- Inspect the source implementation at https://x.com/0xidanlevin/status/2100937437325205568.
- Configure local Jev API credentials and register required tool/model bindings.
- Execute test prompts and observe decision latency and routing outputs.
Why this build matters
Rigorous empirical proof that offloading decision boundaries from frontier models achieves parity at drastic savings.
Reported performance
Reported by authorCost: 112× lower model cost
Latency: ~7 s run time
Author reported 49/49 tasks solved with 112× lower model cost than frontier models using code execution.
Limitations
- Metrics are author-reported from the initial release unless independently verified.
- Requires access to the respective agent framework or runtime environment.
- Generative model execution remains external to the Jev decision step.