HN 日本語サマリー

← 一覧へ戻る
AI・機械学習

Ring-Zero: 兆パラメータへのスケーリングによるゼロRLでの創発的推論

Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning (arxiv.org)

49 pointsby binyu17 コメント

要約

本論文は、人間によるアノテーションなしで検証可能な報酬を用いた強化学習(ゼロRL)の、兆(Trillion)パラメータ規模へのスケーリングに関する研究成果を発表しています。大規模モデルでは、単純なスケーリングでは読みにくさや冗長性、推論深度の不足といった問題が生じますが、本研究では、クリップド重要度サンプリングなどのアルゴリズムとシステム最適化を組み合わせた安定かつ効率的な学習パイプラインを提案しています。実験により、1兆パラメータへのスケーリングがサンプル効率と性能上限を大幅に向上させ、モデルが構造化された推論、自己検証、文脈不安などの高度な認知能力を自律的に獲得することを示しました。

全文翻訳

Computer Science > Computation and Language

arXiv:2607.12395 (cs)

[Submitted on 14 Jul 2026 (v1), last revised 16 Jul 2026 (this version, v2)]

Title:Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning

Authors:Xinyu Tang, Qianggang Cao, Yurou Liu, Yuliang Zhan, Xiaochong Lan, Yifan Li, Yuchen Yan, Han Peng, Zican Dong, Zhenduo Zhang, Tianshu Wang, Xinyu Kong, Zujie Wen, Wayne Xin Zhao, Zhiqiang Zhang, Jun Zhou

View a PDF of the paper titled Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning, by Xinyu Tang and 15 other authors

View PDF HTML (experimental)

Abstract:Reinforcement learning with verifiable rewards without human-annotated data, often referred to as zero RL, has emerged as a powerful paradigm for eliciting chain-of-thought reasoning. However, due to computational constraints, existing studies are largely restricted to small models, leaving the training dynamics and emergent capabilities at a large scale unexplored. To meaningfully explore this frontier, we aim to elicit high-quality reasoning behaviors from the model. However, we find that naive scaling often suffers from poor readability, token redundancy, and a lack of adaptive reasoning depth. To address these challenges, we present a stable and efficient training pipeline, incorporating algorithmic and system optimizations such as clipped importance sampling, training-inference ratio correction, and mixed-precision control. Our experiments offer three key findings that validate the "bitter lesson" of scaling: (1) scaling to 1T parameters significantly enhances sample efficiency and performance ceilings; (2) the training process progresses sequentially through an initial discovery phase followed by a sharpening phase; and (3) the model spontaneously develops advanced cognitive behaviors, including anthropomorphism, structured formatting, self-verification, parallel reasoning, and context anxiety, rendering hand-crafted heuristics redundant. Evaluated on seven mathematical benchmarks, Ring-2.5-1T-Zero achieves competitive performance. Additionally, to assess CoT quality beyond final-answer correctness, we propose a structured evaluation framework across three dimensions: comprehensibility, reproducibility, and efficiency, where our model demonstrates clear advantages in producing structured and concise reasoning traces. By sharing our observed emergent phenomena, we hope to provide the community with deeper insights into scaling behaviors, particularly at the 1-trillion scale.

Subjects:Computation and Language (cs.CL)
Cite as:arXiv:2607.12395 [cs.CL]
(or arXiv:2607.12395v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2607.12395 arXiv-issued DOI via DataCite

Submission history

From: Yuliang Zhan view email
v1 Tue, 14 Jul 2026 06:14:55 UTC (4,310 KB)
v2 Thu, 16 Jul 2026 03:43:34 UTC (4,310 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning, by Xinyu Tang and 15 other authors

[画像: license icon] view license

Additional Features
Current browse context:

cs.CL

< prev   |   next >

new | recent | 2026-07

Change to browse by:

cs

References & Citations

Loading...

BibTeX formatted citation

loading...

Data provided by:

Bookmark

[画像: BibSonomy][画像: Reddit]

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is CORE?)

  • Author
  • Venue
  • Institution
  • Topic

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)