Overview & result
Sniff Test is a prose linter for Markdown and plain text. Countable rules are regular expressions that run locally. Judgment rules, such as a closing sentence that only restates the paragraph, send one paragraph at a time to Jev and get back one probability per rule, and only after you opt in. In the author's own benchmark, the Jev arm caught 63 of 80 judgment faults and flagged 1 of 54 clean paragraphs.
From the original post
A prose linter that sniffs out AI writing tells. Zero dependencies, countable rules plus one judgment model.
How Jev fits in the loop
- Countable rules, written as regular expressions, run locally on Markdown or plain text.
- After you say yes, judgment rules send one paragraph at a time to Jev with each rule's wording.
- Jev returns a probability per rule; 0.7 or above is a flag.
- A reading between 0.4 and 0.6 is reported as no judgment and never becomes a flag or a pass.
How to reproduce
- Inspect the original showcase and source material at https://github.com/DanRWilloughby/snifftest
- Verify the bounded prompt and input candidate schema configured for Jev
- Benchmark decision latency and classification accuracy against baseline models
Why this build matters
The no-judgment band exists because, in the author's words, the model answers about 0.5 on text it cannot read, and a flat middle number would otherwise look like a clean draft.
Limitations
- Benchmark numbers are the author's own eval: one run on 80 faults and 54 clean paragraphs.
- It does not rewrite text; a flag is a sentence for you to fix.
- Judgment rules need a TypeSafe key and send paragraphs to a hosted model.