HN 日本語サマリー

← 一覧へ戻る
AI・機械学習

永続状態機械:INT4インメモリセルによるLLMアテンション

Persistent State Machines: LLM Attention with INT4 In-Memory Cells (zenodo.org)

3 pointsby yusuke_esaka0 コメント

要約

本論文は、永続状態機械(PSM)を用いた大規模言語モデル(LLM)のアテンション演算のための形式的な離散フレームワークを提案しています。計算は静的なインメモリセルにブロードキャストされ、局所的な決定論的状態遷移を評価します。量子化誤差の境界、明示的な有界ロジット仮定下での具体的なマルチフェーズ離散ソフトマックス構成、空間因数分解との決定論的有限オートマトン同等性、および DSPACE(O(n)) への所属に関する完全な数学的証明が提供されます。さらに、FPGA上での実装可能性を評価し、低消費電力とシステムレベルでの互換性を示しています。

全文翻訳

Persistent State Machines: Complete Mathematical Proofs and Vivado Implementation Synthesis (Version 8.0)

Published August 2, 2026 | Version v8

    1. Cosmos Administrative Scrivener Office & Independent Researcher

Description

Persistent State Machines: Complete Mathematical Proofs and Vivado Implementation Synthesis (Version 8.0)

ABSTRACT We present a formal discrete framework for attention operators in Large Language Models via Persistent State Machines (PSMs). Computation is broadcast to stationary in-memory cells that evaluate local deterministic state transitions. Complete mathematical proofs are given for quantization error bounds, a concrete multi-phase discrete Softmax construction under an explicit bounded-logits assumption, deterministic finite-automaton equivalence with spatial factorization, and membership in DSPACE(O(n)).

We validate the implementation feasibility of the architecture on contemporary programmable logic fabric through two distinct evaluation flows:

  1. Low-Power Evaluation (Zynq-7000 xc7z020): A full 1024-cell array (d=128) out-of-context (OOC) block design implementation. By annotating the post-route netlist with a Switching Activity Interchange Format (SAIF) file obtained from a post-route functional simulation, we demonstrate that hardware-level self-activation gating restricts active dynamic switching to sparse cells. The dynamic power of the core logic is estimated below 1.0 mW, yielding a normalized dynamic energy of 3.81 × 10^-5 pJ/op.

  2. System-on-Chip PCIe Integration (UltraScale+ xcvu9p): To verify system-level compatibility, a 256-cell sub-array (representing an integrated 1-head attention system) is integrated within a full SoC. This includes AMBA AXI4 interconnects and an AMD Xilinx PCIe Gen3 x1 Bridge (XDMA v4.2). The complete SoC successfully closed timing at a unified system clock of 62.5 MHz (Worst Negative Slack WNS = +1.854 ns) under the AWS Cloud FPGA Developer environment. The SoC occupies only 0.67% of the device's logic slices and 0.00% of DSP blocks, showing high scaling potential.

Functional simulation with more than one thousand random vectors confirmed bit-exact agreement with a fixed-point software reference. All energy figures are simulation-based tool estimates for the synthesised logic; no physical FPGA board measurement was performed, and system-level external memory energy is strictly excluded.

Japanese Patent Application No. 2026-177318 (Patent Pending)

Files

029_Zenodo_ASMA_Paper_v8_0_FullProofMaster.pdf

Files (17.7 kB)

NameSize
029_Zenodo_ASMA_Paper_v8_0_FullProofMaster.pdf md5:dc7e3ec0a80faaa38e45073a6166073917.7 kBPreview Download