Contrastive Language Models

A System One Model for Fast and Generalizable Decision-Making Jacky Kwok$^{\dagger}$, Hangoo Kang, Tarun Suresh, Jon Saad-Falcon, Marco Pavone Christopher Ré, Azalia Mirhoseini :stanford:Stanford University :nvidia:NVIDIA Research $^{\dagger}$ Project Lead 🗓️ Posted: Sep 23, 2026 <aside> ⚡ New Architecture, Data Recipe, and Scaling Laws We introduce Contrastive Language Models (CLMs), a new class of System One model trained with a contrastive learning objective that connects states and actions. We release CLM-8B, which is pre-trained on 60M Nemotron Q&A pairs, mid-trained on 30M synthetic hard negatives, and post-trained on 1M agentic trajectories. CLM-8B delivers performance comparable to Jev across computer-use, gaming, and tool-calling tasks, while achieving up to 9× lower latency. With lightweight fine-tuning, CLM-8B also sets a new SOTA on challenging agentic coding benchmarks, including DeepSWE (81.6%) and Terminal Bench 2.1 (87.6%). We build an ultra-efficient training and serving infra for CLM by disaggregating states and actions, allowing their embeddings to be cached and reused independently. We establish scaling laws for CLMs and show that the test contrastive loss decreases predictably as a power law in training compute, model size, and dataset size. Try CLM on GitHub: </aside> Overview CLM first trains a state encoder and an action encoder on a large-scale dataset with a contrastive objective (InfoNCE), so that each state is pulled toward the ground truth action that was taken and pushed away from all others. The two encoders then serve directly as a zero-shot action classifier. At deployment, given the current state and a set of candidate actions, CLM scores each action by how well its embedding aligns with the state embedding and selects the highest-scoring action. Dino Run (CLM vs. Jev) Super Mario Demo WikiRacing Demo Zero-shot Evaluation Across computer-use, gaming, and tool-calling tasks, CLM-8B performs on par with Jev while running up to 9× faster. The speedups are most pronounced when the number of candidate actions is large (e.g., WikiRacing) or when actions can be frequently reused across states (e.g., T-Rex Game).