Core takeaway
Diogo Almeida (a core InstructGPT author and key RLHF contributor) explains why ChatGPT's success led the industry to believe autoregressive text generation is the only model form; breaks down how RLCD (Reinforcement Learning from AI Feedback) trains a high-confidence System 1 decision model over a bounded option space; and argues that a two-system architecture is the only economical way for autonomous agents to scale.
Key chapters
00:00Introduction: why OpenAI's technical path has no room for Jev08:15From RLHF to RLCD: what reinforcement learning for bounded decision models really is17:40The weak spot of autoregressive generation: 90% of latency in high-frequency agent loops is wasted26:30The math of softmax normalization over a bounded candidate space38:50Hands-on design of a production two-system (System 1 / System 2) architecture49:10Outlook: the economic tipping point when millions of agents decide in parallel
Key insights
- ChatGPT created a path dependence of treating every problem as "write some text", but an agent is fundamentally a state-machine control loop;
- RLCD training makes Jev's confidence probabilities well calibrated, so its numbers can be used directly as deterministic risk-control thresholds;
- A single decision goes from 1,000+ generated tokens to zero, eliminating the retry loops caused by formatting errors.
- Jev is not meant to replace long chain-of-thought models; complex, open-ended reasoning still belongs to System 2 models;
- If the candidate space is incomplete (no fallback option), the model's confidence distribution can be distorted.
Overview & result
Swyx and Alessio interview Diogo Almeida on why agents require dual-process cognition, the mathematics of RLCD training, and why generative autoregression fails on high-frequency agent loops.
From the original post
Why I couldn't build Jev at OpenAI — Diogo Almeida, TypeSafe Co-founder & CEO. A 58-minute masterclass on RLCD, System 1 cognition, and why agents cannot scale on generative LLMs alone.
How Jev fits in the loop
- Expose the fundamental bottleneck of autoregressive decoding in high-frequency loops.
- Explain how RLCD trains calibrated token-space probabilities over fixed candidate sets.
- Demonstrate the economic scale advantage of dropping prompt tokens to zero.
- Lay out the multi-agent control roadmap combining fast reflexes with slow reasoning.
How to reproduce
- Listen to the 58-minute interview covering RLCD and candidate space normalization.
- Map high-frequency state transitions in your agent loop to bounded Choice/Score calls.
- Compare token consumption before and after dual-system separation.
Why this build matters
The definitive philosophical and engineering foundation explaining why Jev represents a permanent shift toward dual-process AI cognition.
Limitations
- In-depth podcast discussion; conceptual architecture rather than standalone code repo.
- Proprietary pre-training weights and datasets are not open-sourced.
- Best suited for AI system architects and multi-agent engineers.