Redox: Ring Buffers Project Summary

Ring Buffers Project Summary

By Ibuki Omatsu, Anhad Singh, Jacob Lorentzon, Ron Williams on Sunday, October 25, 2026

昨年、Redoxチームは助成金を受け、システムタスクレベルでio_uringライクなリングバッファサポートを開発しました。これは、アプリケーションと大多数のシステムサービス間のリングバッファ通信に向けた第一歩でした。この作業フェーズは9月に完了し、最後のピースが配置されました。

NLnet Foundation、NGI Zero Commons Fund、Horizon Europeの寛大な支援に感謝いたします。NGI Zero Commons Fundは終わりに近づいていますが、NLnetは現在、新しいファンドであるRestackとCode Supply、およびいくつかの小規模テーマファンドからの助成金の申請を受け付けています。2026年11月3日までに応募してください！（その後も2ヶ月ごとに応募可能です）。

チームは素晴らしい仕事をしてくれました。Ibuki、Anhad、Jacobにお祝いを述べ、彼らの時間と労力に感謝します。

サポート可能なシステムサービスとドライバのためのリングバッファ機構の実装に加え、アプリケーション用のio_uringおよびliburingライクなAPIが完成し、デモンストレーションされました。また、ドライバ用の非同期エグゼキュータを含むサポートクレートにもかなりの作業が行われました。

さらに、アプリケーションとそのシステムサービス間の切り替えを大幅に高速化するための、いくつかの非常にエキサイティングなパフォーマンス改善作業も行われました。しかし、技術的な詳細はチームに任せることにします。

– Ron Williams

Ring Buffers Overview

– Anhad Singh

歴史的に、Linuxでの非同期I/OはカーネルのネイティブAIOインターフェースを通じて行われてきました。AIOは主にバッファリングされていない（O_DIRECT）ファイルI/Oに役立ちますが、バッファリングされたリクエストはサイレントにブロックします1。開発者はまた、その複雑さとパフォーマンスについても懸念を表明していました1 2。POSIX AIOも存在しますが、開発者はしばしばその実装を貧弱で遅いと考えています1。これらの問題に対処するため、fibrilsやsyslets/threadletsのようないくつかの代替非同期メカニズムが提案されましたが、メインラインのLinuxカーネルにはマージされませんでした。2019年初頭、Jens Axboeは、効率的で拡張可能で「使いやすく、誤用しにくい」ように設計されたio_uringインターフェースを提案・開発しました。初期の頃、io_uringにはいくつかのセキュリティ問題やメモリリークがありましたが、これらはその後対処されました3。これはLinux 5.1にマージされ、現在広く使用されています。

Linuxのio_uringインターフェースは、ユーザーが1つ以上のI/O操作をカーネルに送信することを可能にし、それらはユーザー空間プロセスをブロックすることなく非同期的に処理されます。マイクロカーネルであるRedoxは、カーネル内ではなく、ユーザー空間のスキームプロセスでI/Oを処理します。各ファイルディスクリプタはそのスキームへのハンドルを格納するため、そのファイルに対するI/Oシステムコールはリクエストをスキームに転送できます。このプロジェクトで開発された非同期I/Oインターフェース、Redox Ringsは、io_uringに似たメカニズムを使用しています。しかし、操作をカーネルに送信するために使用されるio_uringとは異なり、Redox Ringsインターフェースは、アプリケーションがプロセスをブロックすることなく、適切なスキームに直接リクエストを非同期に送信するためにアプリケーションによって使用されます。さらに、このインターフェースは、スキームとアプリケーションの間に別の共有メモリ領域（DMA対応メモリとして要求可能）を持つことで、メモリコピーなしでI/Oリクエストを発行することを可能にします。

移植性のために、Redox Rings APIはliburingのAPIに似るように設計されており、一般的なプロセス間通信にも使用できます。例えば、RedoxFSファイルシステムサービスとNVMed（ストレージドライバ）間の通信に使用されています。このAPIの追加により、Redoxの多くの部分で大幅なパフォーマンス向上が実現しました！

Terminology

A ring buffer is a circular queue backed by an array. The interface makes use of two ring buffers which are shared between the application and the scheme:

• SQ: submission queue, where the application writes requests and the scheme reads them
• CQ: completion queue, where the scheme writes responses and the application reads them
• SQE / CQE: submission queue entry / completion queue entry, entries in the SQ and CQ, respectively
• Buffer pool: A contiguous block of memory, shared between the application and the scheme, from which data buffers are allocated
• Data buffer: For a request, we currently support a single data buffer, which is a slice of the contiguous buffer pool, defined by an offset and length

各キューには正確に1つのプロデューサーとコンシューマーがあるため、各インデックスは一方の側によってのみ変更されます。これにより、共有ロックメカニズムのコストが排除されます。

The “redox-ring API”

– Ibuki Omatsu

We’ve introduced the first ring buffer implementation to Redox OS. Unlike Linux’s io_uring, the submission queue (SQ) and completion queue (CQ) are shared exclusively between userspace processes and are never accessed by the kernel.

redox-rings is a new crate for ring buffer communication between userspace processes, also providing data structures for FFI use cases. It uses futexes as the main synchronization mechanism, with optional pipe file descriptor support.

Based on its FFI support, we’ve introduced the “redox-ring API” in relibc, inspired by Linux’s liburing.

liburing is a library for Linux’s io_uring, providing helpers for setting up and tearing down io_uring instances, as well as a simplified interface for submitting I/O requests.

Similarly, the redox-ring API allows applications to communicate with daemons that support ring buffers using custom protocols.

Key functions

The API provides functions for managing ring buffers, shared memory, and requests.

• redox_ring_queue_init_with_path() initializes a ring buffer connection to a daemon.
• redox_ring_alloc_buf() and redox_ring_alloc_sized() allocate buffers from the shared memory pool.
• redox_ring_get_sqe() obtains a submission queue entry (SQE) for preparing a request.
• redox_ring_submit() submits pending requests to the daemon.
• redox_ring_wait_cqe() waits for a completion queue entry (CQE), blocking if none is available.
• redox_ring_cqe_seen() marks a CQE as processed.

The API also provides functions for registering file descriptors, freeing allocated buffers, and tearing down ring buffer instances.

– Ibuki Omatsu

The redox-rings implementation uses shared memory to exchange SQEs, CQEs, and I/O data between applications and daemons.

During initialization, the SQ, CQ, and data buffer pool are mapped into the application’s address space using the fmap syscall with MAP_SHARED. This allows both sides to access the same memory without copying I/O data between processes.

Buffer allocation

The redox-ring API supports two buffer allocation strategies:

• Fixed-size allocation: Uses a simple allocator that divides the shared memory pool into fixed-size chunks and maintains a list of available offsets.
• Dynamic allocation: Uses the redox-buffer-pool crate to allocate buffers of varying sizes and alignments. The allocator uses B-trees to manage free and occupied memory regions.

Both allocation strategies return a pointer to the allocated buffer and its offset within the shared memory pool.

Once an operation has completed, the application can release its buffer, allowing the memory to be reused.

Session Creation and Management

– Anhad Singh

The redox_ring_queue_init_with_path function opens the scheme ring handle and uses the SYS_CALL system call on it to set up the ring buffer session with the given parameters. This returns file descriptors for the SQ, CQ, and shared buffer pool, which are mapped into the application’s address space. The application can then use the queues directly to submit requests and receive completions. On the scheme side, the worker task waits for entries in the SQ, processes them asynchronously and notifies the application on completion.

Example: Writing to RamFS

– Ibuki Omatsu

Ring buffer support was added to RamFS and NVMed. We also added tests that communicate with these storage daemons through the redox-ring API.

For example, writing data to RamFS involves the following steps:

1. Open the target file and initialize a ring buffer instance to RamFS using redox_ring_queue_init_with_path().
2. Register the file descriptor so that subsequent requests can reference it by index.
3. Allocate a shared memory buffer and write the data into it.
4. Obtain an SQE and specify the write operation, file index, buffer offset, data length, and file offset.
5. Submit the prepared SQEs using redox_ring_submit(). Multiple requests can be submitted in a single batch.
6. Wait for CQEs, check the results, and release the buffers associated with completed requests.

Once all operations have completed, the application tears down the ring buffer connection.

Async Executor

– Anhad Singh

We developed an async executor for device drivers, with a single interrupt vector and an SQ/CQ pair per core architecture. A single core is used at the moment, and multi-thread-per-core support is in progress. Furthermore, we updated NVMe driver to use the async executor, and more devices such as virtio are expected to follow.

With the executor, schemes that support Redox Rings can handle each I/O request in its own async task, with a worker thread spawned for each session between the application and the scheme. On teardown, the worker task is stopped, pending I/O operations are cancelled and shared resources are cleaned up.

［画像: Ring Buffers Benchmark］

Figure 1: Comparison between the redox_syscall and redox_ring benchmarks, showing that redox_ring is 14-15x faster. The redox_syscall benchmark performs I/O via synchronous system calls, while the redox_ring benchmark uses the new Redox Ring interface.

Kernel and IPC optimizations

– Jacob Lorentzon

While it has substantially improved over the years, IPC is still quite slow on Redox, and this is important to optimize as ring buffers can only offset this overhead to a certain extent. This has been a focus of our work to improve system performance.

Infrastructure

The acid test suite for Redox was expanded to include more targeted benchmarks that measure IPC latency, in particular acid ipc_latency_simultaneous, and also acid heavy_forking. Support was also added for repeating measurements and collecting more proper statistics (although still less feature-complete than e.g. hyperfine), making it easier to know whether changes in metrics are statistically significant and by how much.

Miscellaneous refactors

Various single-digit speedups were attained from kernel refactors, such as eliminating redundant locking, hashmap lookups, and by adding a lower-overhead call for IPC, similar to seL4’s “reply-wait”.

Fast path switching without vector registers

Another source of overhead, from the flamegraphs, was found to be XSAVE and XRSTOR, a few hundred cycles combined (tested on an AMD Ryzen 9 5950X). These instructions save/restore vector registers, which is ~600 bytes on the CPU tested, and likely similar on other CPUs with AVX2 (which only userspace uses).

The optimization made here is that the kernel zeroes the vector registers if the caller and callee entered the kernel through syscalls, rather than saving/restoring (using vzeroall, ldmxcsr etc. as required for correctness).

Context switching can be triggered both by preemption, i.e. timer interrupts, and voluntarily from syscalls. After Redox destabilized its syscall ABI, instead preferring the libredox symbols provided by relibc, all syscalls will first go through a function call. Since the System V ABI clobbers all vector registers for all function calls, it is almost zero-cost to also define Redox’s syscall ABI to do so. Luckily, the generated assembly code for relibc was almost identical.

Direct switching

A major bottleneck during IPC is invoking the scheduler, as can be seen from various flamegraphs. The scheduler needs to access multiple data structures, primarily BTreeMap that takes up a large fraction of the overhead, and it also needs to acquire lots of locks, often blocking.

Direct context switching was implemented in kernel MR 694, resulting in a 15-20% IPC latency reduction. It separates the current context (reference) into the scheduled context and the running context, where the former is the context owning the current timeslice, and the latter is the context whose registers/memory/file table is currently active. When direct switching has not occurred, these always coincide. Consequently, a client process calling a scheme (typically using our SYS_CALL syscall) can completely bypass the scheduler and switch directly to the scheme, if ready, where the kernel will then prioritize returning to the scheduled context the next time the scheme yields. This mimics the behavior of how complex syscalls are handled in monolithic kernels, and should also be more cache-efficient, among other efficiencies.

Huge pages

The kernel previously only supported 4k pages. This is still the case on non-x86, and for all userspace cases, but now the kernel physmap will be mapped as 2M when possible. TLB usage can thus theoretically be reduced by up to 512x for heavy CoW, and in practice, the highest obtained L1DTLB-miss rate from acid heavy_forking was -75%.

Future improvements

– Jacob Lorentzon

IPC roundtrip latency has been reduced from ~5800 cycles to ~2700 cycles, but this is still high compared to fast µkernels like seL4. Additional optimizations could be to reduce locking, and reduce the syscall entry overhead, but primarily to simplify the IPC primitive and remove possibly unnecessary parts such as request buffers. While not as applicable for nonlegacy applications, automatic page-borrowing of clients’ buffers (in UserScheme) is also a major source of overhead and in a benchmark similar to dd bs=4k if=/dev/zero of=/dev/null, the overhead is at least ~32k cycles per call. However, while the existing benchmarks are useful as an approximate lower bound on latency, more investigation will be required to determine the distribution of overheads for more realistic workloads, both with and without userspace ring buffers.

---

1. https://kernel.dk/io_uring.pdf ↩︎ ↩︎ ↩︎
2. https://lwn.net/Articles/671649/ ↩︎
3. https://github.com/axboe/liburing/discussions/1047#discussioncomment-8374809 ↩︎