DEMO / RECORDING
Show the original post (Chinese)
拿 Jev 做搜索重排，我先泼一盆冷水：单独用，它没打赢向量检索
TypeSafe 的 Jev 这阵子很火，一堆项目拿它做重排。我们在 Agent Skills Hub 的 33,047 条目录上认真测了一次，164 条中英文真实查询，9,831 对分级标注，整套只花了 2.6 美元
三个结论
01｜单独重排，约等于没赢
Jev 重排 bge-m3 的前 30 条，NDCG@10 只多了 0.012，置信区间跨过零。MRR 和前三命中率倒是涨得明显，它很会把最强的那一条顶到第一，后面几条基本是重新洗牌
02｜裁判偏差，被我们量出来了
Jev 自己也参与了打标注，这就是循环
只用 Jev 当裁判，它领先 0.053
两个裁判合并，领先 0.012
只用 Haiku 当裁判，反而落后 0.028
同一组比较，换个裁判结论直接翻面。所有涉及 Jev 的结论，我们只认 Haiku 那一列
03｜真正稳赢的是融合
把 Jev 和 bge-m3 的排序做 RRF 融合，NDCG@10 到 0.864，比纯向量高 0.064 到 0.116，三种裁判下都成立。代价是每次查询多一次 API 调用，约 0.0002 美元
顺便暴露了我们自己的问题：Hub 线上 CLI 用的关键词排序只有 0.609，短板是召回。相关结果有一半压根没进候选，后面怎么重排都救不回来
数据、标注、每条查询的得分全部开源，不用 API key 就能复现打分
How Jev fits in the loop
- Agent collects runtime context or input state from external sources.
- Bounded decision options are formatted into Jev System One questions.
- Jev scores each candidate option and returns a probability for it.
- Deterministic code executes the selected action or routes to specialized models.
How to reproduce
- Read the author's original post on X at https://x.com/GoSailGlobal/status/2100877682972258619.
- Inspect the described workflow architecture and bounded decision points.
- Test Jev System One decision queries against the provided task context.
Why this build matters
Demonstrates real-world Jev Skills experimentation by @GoSailGlobal with verified community reach of 17.9k views.
Limitations
- Implementation details are based on the author's public post on X.
- Performance figures and benchmarks are author-reported community claims.
- Production deployments may require custom calibration and policy thresholds.