COMMUNITY OBSERVEDMetric claims: AUTHOR REPORTEDSOURCE AUDITED🤖 Jev Agent

Stencil QA Agent

A QA loop where Jev selects browser actions while a frontier model plans and vision verifies.

Overview & result

Dan Leshem reports rebuilding Stencil's QA agent around Jev: a frontier model plans the test, Jev chooses browser actions, and vision checks the result. The shown run signs in, opens an app, triggers a paywall, checks a spacing fix, and passes five checkpoints in 28.3 seconds with a screenshot and trace for each step.

From the original post

we rebuilt a QA agent around @typesafeai’s Jev at Stencil.

a frontier model plans the test. Jev chooses browser actions. vision handles the visual checks.

here it signs in, opens an app, triggers the paywall, and checks a spacing fix.

5 checkpoints passed in 28.3 seconds. every checkpoint has a screenshot, every decision has a trace.

i can see Jev driving decisions across our software factory: which issues to prioritize, what to test, when to retry, and when a change needs human review.

— Dan Leshem

How Jev fits in the loop

  1. The frontier model plans the QA scenario.
  2. Jev selects the next bounded browser action.
  3. Vision checks the rendered result while the runtime decides whether to continue, retry, prioritize, or request human review.

How to reproduce

  1. Open the original Stencil QA post and inspect the attached video as the primary evidence.
  2. Model the loop as frontier-model planning, Jev browser-action choice, and vision-based verification.
  3. Treat the reported timing as a target to remeasure; do not present it as an independent benchmark.

Why this build matters

This separates repeated action/evaluation decisions from generation and visual inspection inside a browser QA loop.

Limitations

  • The post does not link a public repository or a runnable reproduction.
  • The 28.3-second result is an author-reported run, not a JevForAgents measurement.

Patterns