Overview & result
Tenbin is a repository for using TypeSafe's System One API (Jev) from a coding agent at design time. It combines documentation, a stdio MCP server (question lint, evaluation, batch evaluation for threshold calibration, rerank, taxonomy exploration, cost guard) and an agent skill for decomposing a judgment into Choice, Score and Noul questions, measuring it on labelled data and putting thresholds in code.
From the original post
MCP server and agent skill for the TypeSafe AI System One API (Jev): decompose a judgment into Choice / Score / Noul questions, lint them, measure on labelled data, and put calibrated thresholds in code
How Jev fits in the loop
- You ask your coding agent for a judgment your code needs.
- The skill decomposes it into atomic Choice, Score and Noul questions and lints them offline.
- The MCP server evaluates the questions on your labelled rows and reports accuracy per confidence band.
- Thresholds are written to one file with the measurement date, alongside production code that calls the SDK.
How to reproduce
- Inspect the original showcase and source material at https://github.com/simota/tenbin
- Verify the bounded prompt and input candidate schema configured for Jev
- Benchmark decision latency and classification accuracy against baseline models
Why this build matters
Thresholds come from measurement on your own labelled data, and cookbook thresholds are marked provisional until measured.
Limitations
- It is a design-time tool for a coding agent, not a runtime service.
- Calibration requires labelled data.
- Design and question-generation steps need no API key, but evaluation calls the TypeSafe API.