Overview & result
A minimal coding agent that filters every tool result through Jev before the main model sees it, with an A/B harness that runs the same tasks with filtering on and off. The author reports a negative result: on 49 scored TB4 pairs, the Jev arm passed 2 tasks against 5 without it, and used 11.78% more input tokens and 8.17% more estimated model cost. The README says the experiment is closed.
From the original post
A coding agent that filters every tool result through Jev before the model sees it, with an A/B harness measuring pass@1 and cost against the unfiltered control
How Jev fits in the loop
- The main model works with three tools: bash, read and apply_patch.
- Every tool result except read is split into chunks of about 2,000 characters on line boundaries.
- Jev answers one yes/no question per chunk: does it contain information the agent needs for this step or task? Chunks above 0.5 are kept.
- The raw output is always stored on disk, and read can retrieve it.
How to reproduce
- Inspect the original showcase and source material at https://github.com/Astro-Han/jev-harness
- Verify the bounded prompt and input candidate schema configured for Jev
- Benchmark decision latency and classification accuracy against baseline models
Why this build matters
It is a published negative result: the author measured the idea with an A/B harness, found it did not pay off, and closed the experiment.
Limitations
- Each evaluation used one run per arm and a single main model; the README says neither establishes general effectiveness.
- Trajectory inspection found unbounded-output failures and environment limitations.
- A closed prototype, not a maintained tool.