Overview & result
Switchloom assigns Codex capabilities to persistent model tasks and can optionally use Jev for routing. The author reports that two Pokedex pilots did not show a Jev advantage at comparable quality; in round two, the Jev team reached the time limit while Astra alone passed all 14 checks.
From the original post
Experimental. I currently recommend against adopting this routing workflow. I built Switchloom to see whether mixing models could get me the same quality for less money. My benchmarks did not deliver that result. In the second Pokédex run, the team with Jev cost almost as much as Astra alone and hit the time limit before final acceptance. Astra finished and passed all 14 independent functional checks.
How Jev fits in the loop
- Assign capabilities and model slots in the Switchloom workflow.
- In optional Jev mode, use the TypeSafe decision to prepare routing and handoff arguments.
- Codex creates and runs the persistent tasks; Switchloom does not execute the coding work.
- Compare Jev and non-Jev runs using the published benchmark artifacts.
How to reproduce
- Read the repository's setup and Pokedex benchmark limitations before use.
- Compare the same Codex task with one model, a team without Jev, and a team with Jev.
- Inspect artifacts, costs, time limits, and independent functional checks rather than routing output alone.
Why this build matters
A useful counterexample: adding a Jev router did not improve this author's two pilots, so routing overhead and end-to-end quality need measurement.
Limitations
- The author recommends against adopting this workflow based on two pilots of one application.
- The 1.0 desktop workflow is described as an unpublished local preview; representative desktop measurements remain open.