Overview & result
Jev Model Router classifies each query against two or three developer-configured model tiers. Code applies confidence thresholds to choose the tier and uses a configured fallback if the TypeSafe API fails; no measured cost-saving percentage is reported.
From the original post
jev-model-router routes each query to the cheapest LLM tier that can actually handle it, using TypeSafe's Jev to classify how demanding a query is — fast and cheap, without spending an LLM call on the routing decision itself.
How Jev fits in the loop
- Pass the query and ordered model descriptions to Jev.
- Apply configured confidence thresholds to Jev's tier probabilities.
- Return the selected model name, tier, probabilities, and fallback status.
- Let the host application call the chosen model.
How to reproduce
- Install jev-model-router and define an ordered two- or three-model routing table.
- Submit varied benchmark prompts ranging from trivial formatting to complex code refactoring.
- Observe routing accuracy, downstream response quality, and total token savings.
Why this build matters
Provides a reproducible example of separating a cheap bounded routing decision from downstream text generation.
Limitations
- Routing quality depends on the descriptions and confidence thresholds chosen by the integrator.
- The repository documents a fallback model for API failures, not an automatic quality-based retry policy.
- Does not perform the underlying text generation.