Overview & result
A Node app that controls a headed Chromium window by voice. Speech streams word by word to a small server, and on every partial transcript one Jev request asks about a dozen typed questions: intent, target element, site, whether the command is complete, whether it is addressed to the app, and whether it is destructive. Code then decides to act, wait, ask or ignore. The author reports 250–350 ms per request.
From the original post
Control a real browser by voice. Jev (TypeSafe System One) decides intent + target in ~300 ms per spoken word; Playwright acts — often before you finish the sentence.
How Jev fits in the loop
- Speech streams word by word from the browser's Web Speech API to a small Node server.
- On every partial transcript, one Jev request asks about a dozen typed questions: intent, target element, site, completeness, whether it is addressed to the app, whether it is destructive.
- Code decides to act, wait, ask or ignore based on the probabilities.
- Search queries, typed text and URLs are extracted as candidate spans by code; Jev only picks one, and Playwright acts.
How to reproduce
- Inspect the original showcase and source material at https://github.com/moritzkremb/jev-voice-browser
- Verify the bounded prompt and input candidate schema configured for Jev
- Benchmark decision latency and classification accuracy against baseline models
Why this build matters
Because it asks on every partial transcript, it can act before the sentence is finished.
Limitations
- The Web Speech API works only in Chrome or Edge and sends audio to Google.
- One action per utterance; the element snapshot is capped at 100 items and does not see inside iframes.
- Sites with heavy bot protection may not render results.