Overview & result
Voice and typed computer use for macOS. You say or type what you want, Jev picks the next on-screen action, and macOS performs it. The app takes no screenshots: it reads the screen through the Accessibility tree. Setup stores your TypeSafe API key in the Keychain and asks for Accessibility, microphone and speech permissions.
From the original post
Say it, and your Mac does it. A computer-use harness on Jev that reads the screen through Accessibility. Fast, no vision model
How Jev fits in the loop
- Read: the app walks the front app's Accessibility tree, where every element describes its role, name, value and actions.
- Choose: one request to Jev carries the goal, the numbered targets and the last ten actions; Jev selects an operation and a target.
- Act: press, select, type, open a menu, press a key, scroll or arrange windows.
- Check: read the screen again, report the real effect and repeat until DONE, BLOCKED or WAIT.
How to reproduce
- Inspect the original showcase and source material at https://github.com/savka777/jev-use
- Verify the bounded prompt and input candidate schema configured for Jev
- Benchmark decision latency and classification accuracy against baseline models
Why this build matters
It needs no screenshots and no per-app code, and typed text is always a span of what you said, never generated.
Limitations
- Requires macOS 14.2+ and Xcode to build.
- The author gives about 0.3–1.5 s per step, with no task success rates.
- It depends on apps exposing a useful Accessibility tree.