Automatic recovery of crashed agent state with checkpointd

To solve this issue, we’ve created checkpointd. It runs stateful, sandboxed agents on Kubernetes using Agent Substrate, with automatic crash recovery of agent’s state from memory checkpoints.

GitHub - ktock/checkpointd: Checkpoint-based Durable Execution of Agents on Kubernetes

Each agent’s memory is transparently checkpointed and a crashed agent is automatically resumed from where it left off. It uses the A2A protocol for agent-to-agent communication with our custom transport and the standard a2a-go framework can be used to write agents.

Checkpointd started as a fork of Agent Executor (AX) and is built on Agent Substrate, with added support for crash recovery of A2A agents’ state. checkpointd is currently experimental.

When does it checkpoint and resume an agent?

checkpointd checkpoints and resumes agents every time they send or receive a message.

For example, the following pseudocode describes a chat agent that calls an LLM to infer a response to the client’s prompt (callLLM), asks another agent for a second opinion (askReviewerAgent) and eventually returns.

Checkpointd checkpoints and suspends this agent when it sends a message to another agent (askReviewerAgent). When this agent receives a response, checkpointd resumes it. If it crashes before sending the next message or return, it’s recovered from the checkpoint so that its progress isn’t lost.

How does it work?

checkpointd relies on Agent Substrate to checkpoint, resume and sandbox agents. checkpointd itself works as a message relay among A2A agents, controlling the timing of checkpointing and resumption.

checkpointd executes agents in a collaborative manner. Conceptually, checkpointd repeats the following steps for each message.

1. Resume the message’s target agent and pass the message to the agent.
2. When this agent returns a new message (targeting another agent), checkpointd checkpoints and suspends the agent.
3. Persist this new message to the database so it survives checkpointd’s own crashes. Proceed to the next turn for this message.

If the agent crashes before returning a new message, it is recovered from the last checkpoint. If checkpointd itself crashes, it restores the last message from the database and retries the last turn.

Crash recovery demo of a chat agent

This section describes a demo on KinD, showing that a crashed agent is recovered while preserving its state. Prerequisites are Docker, kind, kubectl, Go and git.

First, clone the checkpointd repository.

git clone https://github.com/ktock/checkpointd
cd checkpointd

Start a demo KinD cluster that runs checkpointd and the following agents.

• Chat agent: An agent the client can talk to. It asks the reviewer agent for a second opinion on each response.
• Reviewer agent: An agent that gives the chat agent a second opinion.

NOTE: the demo downloads Qwen3 1.7B quantized by Unsloth (867MB).

./examples/agent-crash-demo/setup.sh

demo.sh starts an interactive multi-turn conversation with the chat agent.

./examples/agent-crash-demo/demo.sh

You can input a message whenever a prompt (>) appears. The chat agent has roughly a 50% chance of crashing on each turn. When it crashes, its memory including the earlier conversation history is recovered from a checkpoint.

In the following example, the agent crashed while answering “What is your name?” which was the second input in the conversation. It then recovered its state including the earlier “Hello!” greeting.

> What is your name?
[agent-crash-demo] running: /tmp/checkpointd-agent-crash-demo-a2a-cli.of6077et/a2a --transport jsonrpc --timeout 240s send http://127.0.0.1:39849/agents/chat-agent/ -o json --task 01a11a5d-637e-7acc-ba46-ef7ff8398aed --tenant 7dec36a4b3224048 What is your name?
[agent-crash-demo] Detected crash of chat-agent: "panic: self-destructing agent-crash-demo"
  reply:
    Assistant: I am a language model named Llama. I help users with various tasks, like answering questions, writing, and more. Let me know how I can assist you!
    Reviewer: The response is informative and helpful, but it could be more engaging by addressing the user directly and asking for their needs. A more personalized approach would enhance the interaction.
    
    --- conversation log from process memory ---
    [0]
    User: Hello!
    Assistant: Hello! How can I help you today?
    Reviewer: The response is appropriate and polite, but it could be more personalized by addressing the user directly. A more engaging follow-up might include a question to encourage further interaction.
    ------------------------
    [1]
    User: What is your name?
    Assistant: I am a language model named Llama. I help users with various tasks, like answering questions, writing, and more. Let me know how I can assist you!
    Reviewer: The response is informative and helpful, but it could be more engaging by addressing the user directly and asking for their needs. A more personalized approach would enhance the interaction.

Finally, the following command cleans up the cluster.

./examples/agent-crash-demo/cleanup.sh

checkpointd: Automatic Crash Recovery of Stateful Agents from Memory Checkpoints on Kubernetes was originally published in nttlabs on Medium, where people are continuing the conversation by highlighting and responding to this story.

• Faster image pulling and pushing
• Support for docker run --net=host
• Support for localhost registries
• Source IP propagation without the legacy slirp4netns dependency

What is Rootless mode?

Rootless mode means running the entire Docker daemon (not just containers) as a non-root user, for protecting the host from potential Docker vulnerabilities and misconfigurations. Even if an attacker escapes from a container, they can access only the files and processes available to the non-root daemon user.

Rootless mode itself is not new; it was originally implemented in 2018 and has been merged into Docker since v19.03 (2019).

RootlessモードでDockerをより安全にする [DockerCon発表レポート]

Getting started

To get started with Rootless Docker, install the docker-ce-rootless-extras package and run dockerd-rootless-setuptool.sh install as a non-root user.

# See https://docs.docker.com/engine/install for how to configure apt
sudo apt install docker-ce-rootless-extras

dockerd-rootless-setuptool.sh install

See also:

• https://docs.docker.com/engine/security/rootless/
• https://rootlesscontaine.rs/

Improvements in Docker v29.5

Rootless Docker had been notorious for its limitations in networking, as the entire daemon was encapsulated in a network namespace (NetNS) associated with a user-mode TCP/IP stack such as slirp4netns.

Such limitations included:

• Poor throughput of image pulling and pushing (typically less than 10 Gbps)
• Lack of support for docker run --net=host
• Lack of support for localhost registries ( docker pull localhost:PORT/IMAGE)

These limitations have been resolved in Docker v29.5, by moving the daemon out of the NetNS associated with user-mode TCP/IP stack.

Notably, support for docker run --net=host should be highly useful, as it allows containers to bypass the overhead of the user-mode TCP/IP entirely. It should be still noted that --net=host carries the security risk of exposing abstract UNIX sockets to the containers, however, it is less likely catastrophic in the case of rootless mode. This concern can be even alleviated by specifying --user=SUBUSER in conjunction.

[CVE-2020–15257] Don’t use --net=host . Don’t use spec.hostNetwork .

Elimination of slirp4netns dependency

Besides, this release also replaces slirp4netns with gvisor-tap-vsock in the default setup, as slirp4netns is based on very old and potentially unsafe C code dating back to the 1990s. In contrast, gvisor-tap-vsock is written in pure Go and expected to have fewer potential vulnerabilities, although it is still not completely free from unsafe code.

In prior releases of Rootless Docker, users often specified an environment variable DOCKERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER=slirp4netns to switch the port driver of RootlessKit from builtin to slirp4netns for enabling source IP propagation in port forwarding ( docker run -p ). Otherwise, the source IP information in TCP packets was always forged to have the IP of Docker’s bridge interface (typically 172.17.0.1).

In Docker v29.5, users do not need to specify DOCKERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER , however, they have to disable userland-proxy for enabling source IP propagation:

mkdir -p ~/.config/docker
echo '{"userland-proxy": false}' >~/.config/docker/daemon.json
systemctl --user restart docker

The userland-proxy is planned to be disabled by default in a future release of Docker:

Disable Userland proxy by default · Issue #14856 · moby/moby

Also, depending on the host configuration, users may need to load the br_netfilter kernel module:

sudo tee /etc/modules-load.d/docker.conf <<EOF >/dev/null
br_netfilter
EOF
sudo systemctl restart systemd-modules-load.service

NTT is hiring!

We at NTT are looking for engineers who work in Open Source communities in the fields of containers, etc. Visit <https://www.rd.ntt/e/sic/recruit/> to see how to join us.

私たちNTTは、コンテナなどの領域のオープンソースコミュニティで、共に活動する仲間を募集しています。ぜひ弊社採用情報ページをご覧ください: <https://www.rd.ntt/sic/recruit/>

Improvements to Rootless mode in Docker v29.5 was originally published in nttlabs on Medium, where people are continuing the conversation by highlighting and responding to this story.

This means third-party package maintainers no longer need to maintain their own web servers for hosting apt packages. They still have to maintain their container image registry, but it is already offered by GitHub for free.

Note: "OCI" in this article refers to the “Open Container Initiative”, not to “Oracle Cloud Infrastructure”.

Example (for package consumers)

The following example installs the hello-apt-transport-oci package from the oci://ghcr.io/akihirosuda/apt-transport-oci-examples:latest image, which is built from the GitHub repository https://github.com/AkihiroSuda/apt-transport-oci-examples .

First, install the apt-transport-oci plugin:

sudo apt install apt-transport-oci

Then create /etc/apt/sources.list.d/oci.sources with the following content:

Types: deb
URIs: oci://ghcr.io/akihirosuda/apt-transport-oci-examples:latest
Suites: stable
Components: main
Signed-By: /etc/apt/keyrings/apt-transport-oci-examples.gpg

Download the GPG key:

curl -fsSL https://raw.githubusercontent.com/AkihiroSuda/apt-transport-oci-examples/refs/heads/master/apt-transport-oci-examples.gpg \
  | sudo gpg --dearmor -o /etc/apt/keyrings/apt-transport-oci-examples.gpg

Confirm the signature:

$ gpg --show-keys --with-fingerprint /etc/apt/keyrings/apt-transport-oci-examples.gpg 
pub   ed25519 2026-04-21 [SC] [expires: 2029-04-20]
      E26B 12C8 C96A 4E4B CDF5  517A 3EB0 4A34 581C DAF6
uid                      Akihiro Suda, on behalf of apt-transport-oci-examples <akihiro.suda.cz@hco.ntt.co.jp>
sub   cv25519 2026-04-21 [E]

Update the apt cache and install the hello-apt-transport-oci package:

sudo apt update
sudo apt install hello-apt-transport-oci

Confirm it works:

$ hello-apt-transport-oci
Hello, apt-transport-oci

Example (for package maintainers)

Packaging dpkg

A dpkg file can be created using the traditional dpkg-deb command with the file tree to be packaged and the DEBIAN/control metadata file as follows:

$ tree hello-apt-transport-oci/
hello-apt-transport-oci/
├── DEBIAN
│   └── control
└── usr
    └── bin
        └── hello-apt-transport-oci

4 directories, 2 files

$ cat hello-apt-transport-oci/DEBIAN/control 
Package: hello-apt-transport-oci
Version: 0.1
Architecture: all
Maintainer: example@example.com
Description: hello apt-transport-oci

$ dpkg-deb --build --root-owner-group hello-apt-transport-oci hello-apt-transport-oci_0.1_all.deb
dpkg-deb: building package 'hello-apt-transport-oci' in 'hello-apt-transport-oci_0.1_all.deb'.

There are also several package building tools. Notably, Dalec is useful for Docker users, as it is implemented as a custom syntax for Dockerfiles: https://project-dalec.github.io/dalec/quickstart

Creating an APT tree

aptly is convenient for creating the APT repository tree:

sudo apt install aptly

aptly repo create hello-apt-transport-oci
aptly repo add hello-apt-transport-oci hello-apt-transport-oci_0.1_all.deb
aptly publish repo -distribution=stable -architectures=all,amd64,arm64 hello-apt-transport-oci

The repository data will be locally published on ~/.aptly/public :

$ tree ~/.aptly/public
/home/USER/.aptly/public
├── dists
│   └── stable
│       ├── Contents-all.gz
│       ├── Contents-amd64.gz
│       ├── Contents-arm64.gz
│       ├── InRelease
│       ├── main
│       │   ├── binary-all
│       │   │   ├── Packages
│       │   │   ├── Packages.bz2
│       │   │   ├── Packages.gz
│       │   │   └── Release
│       │   ├── binary-amd64
│       │   │   ├── Packages
│       │   │   ├── Packages.bz2
│       │   │   ├── Packages.gz
│       │   │   └── Release
│       │   ├── binary-arm64
│       │   │   ├── Packages
│       │   │   ├── Packages.bz2
│       │   │   ├── Packages.gz
│       │   │   └── Release
│       │   ├── Contents-all.gz
│       │   ├── Contents-amd64.gz
│       │   └── Contents-arm64.gz
│       ├── Release
│       └── Release.gpg
└── pool
    └── main
        └── h
            └── hello-apt-transport-oci
                └── hello-apt-transport-oci_0.1_all.deb

11 directories, 22 files

Pushing to a registry

Use ORAS (OCI Registry As Storage) to push an APT tree to a registry:

sudo apt install oras

cd ~/.aptly/public
find . -type f -printf "%p:application/octet-stream\n" \
  | xargs oras push ghcr.io/USERNAME/hello-apt-transport:latest

See https://github.com/AkihiroSuda/apt-transport-oci-examples for further details.

FAQ: Do I need to use containers?

No. The topic is about using the distribution protocol that has been used for containers, not about using containers themselves.

FAQ: Why use container image registries?

Because GitHub Container Registry is free in both storage size and bandwidth:

Billing for container image storage: Container image storage and bandwidth for the Container registry is currently free.

https://docs.github.com/en/billing/concepts/product-billing/github-packages#free-use-of-github-packages

It still has a few limitations, but those limitations are very loose and almost negligible:

・The Container registry has a 10 GB size limit for each layer.
・The Container registry has a 10 minute timeout limit for uploads.

https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#troubleshooting

For these benefits, Homebrew has also been using GitHub Container Registry since 2021.

Why not use GitHub Packages?

Because GitHub Packages does not support APT. The service has been focusing on language package managers such as NPM and Maven so far.

Why not use GitHub Pages?

Because GitHub Pages is intended to be used for Web pages, not for serving arbitrary HTTP(S) content such as APT packages. For that reason, it comes with relatively tight usage limits:

・Published GitHub Pages sites may be no larger than 1 GB.
[...]
・GitHub Pages sites have a soft bandwidth limit of 100 GB per month.

https://docs.github.com/en/pages/getting-started-with-github-pages/github-pages-limits#usage-limits

FAQ: What about DNF?

I didn't support DNF when I wrote the apt-transport-oci plugin in 2021, because DNF at that time didn't appear to have a plugin system that is as flexible as APT.

However, the situation has changed with the release of DNF5 (Fedora 41). Luiz Carvalho at Red Hat recently implemented an experimental DNF5 plugin that enables OCI transport in the same way as apt-transport-oci: https://github.com/lcarva/libdnf5-oci-plugin . I hope that libdnf5-oci-plugin will eventually be included in Fedora, RHEL, and similar distributions.

NTT is hiring!

We at NTT are looking for engineers who work in Open Source communities in the fields of containers, etc. Visit <https://www.rd.ntt/e/sic/recruit/> to see how to join us.

私たちNTTは、コンテナなどの領域のオープンソースコミュニティで、共に活動する仲間を募集しています。ぜひ弊社採用情報ページをご覧ください: <https://www.rd.ntt/sic/recruit/>

Ubuntu 26.04 can install APT packages from GitHub Container Registry was originally published in nttlabs on Medium, where people are continuing the conversation by highlighting and responding to this story.

This article recaps my poster session, “Reimagining Auto-Instrumentation: Prototyping Distributed Tracing Without Header Propagation on K8s” presented at KubeCon + CloudNativeCon Europe 2026.

Introduction

Distributed tracing is typically achieved through header propagation, which passes trace context in request headers so that different services can link their spans into a single distributed trace. Header propagation can be implemented through either manual instrumentation or auto-instrumentation. With manual instrumentation, you embed header propagation logic directly in application code, which incurs high implementation costs and tight coupling. With auto-instrumentation, header propagation is handled below the application layer, so you don’t have to change the application code. However, there are some limitations in terms of supported languages, libraries, or protocols. Furthermore, when receiving and sending are handled by different threads, context loss between threads makes the header propagation approach fundamentally difficult to apply. As long as header propagation is used, removing all such limitations is difficult; therefore, a new approach to achieving auto-instrumentation is necessary.

To address these limitations, auto-instrumentation without header propagation, which infers span relationships, has received attention.

A representative method for this approach is TraceWeaver (SIGCOMM’24). First, it constructs delay distributions between in-span and out-span events. Second, it scores each candidate mapping using these distributions. Third, it selects the most probable mapping.

Toward a Broader Scope for TraceWeaver

However, existing implementations of auto-instrumentation without header propagation do not cover some important cases. First, they assume a request-response communication model and do not handle pipeline architectures, where stages process requests sequentially without request-response pairing. Second, they do not provide a concrete deployment strategy for Kubernetes environments.

In this study, we extend TraceWeaver through the following two tasks. In Task 1, we extend the mapping method for pipeline architectures because demand for efficient pipeline processing platforms is expected to grow, driven by applications such as AI computing and IoT data processing. In Task 2, we consider a deployment configuration for applications on Kubernetes. Specifically, we design a method to capture request data from pods without changing application code.

Proposed Method and Configuration

In this work, we assume a non-branching pipeline, where requests may be dropped at bottleneck stages. When a request is dropped at an intermediate stage, the corresponding span is missing from the trace, resulting in span loss.

To support this pipeline setting, we change the span content and reformulate the mapping problem. In TraceWeaver, each span contains both a request and a response. In contrast, in our setting, each span contains either a request or a response.

Our implementation follows the same iterative framework as TraceWeaver — (1) build the delay distribution, (2) score each candidate mapping — but replaces the optimization in Step (3).

TraceWeaver solves a Maximum Independent Set (MIS) problem to find the best mapping. Because solving MIS on a large graph is computationally expensive, it divides spans into batches. However, batching requires knowing how many skip spans to add to each batch to compensate for span loss. In TraceWeaver’s microservice setting, in-spans and out-spans have a temporal nesting structure, which allows estimating the number of skip spans per batch. In pipeline architectures, this nesting structure does not exist, so the required number of skip spans per batch cannot be determined before mapping.

To avoid this problem, we reformulate Step (3) as a linear sum assignment (LSA) problem, which can be solved using a single N × N cost matrix without batching. Each element represents the likelihood that a given in-span corresponds to a given out-span. When span loss occurs, the number of in-spans and out-spans differs, so the system introduces skip spans — virtual spans that pad the shorter side to form a square matrix. For mappings involving skip spans, a common value (the minimum likelihood in the matrix) is assigned. The LSA solver then selects N elements — one per row and one per column — that maximize the total likelihood.

For the configuration to be used with applications on Kubernetes, we design a setup that uses OpenTelemetry eBPF Instrumentation (OBI) as the request capture tool. Since our approach does not rely on OBI’s context propagation mechanism, there are few protocol limitations. It works by simply specifying the target process to monitor, requiring no changes to application code. For this prototype, we use Jaeger with in-memory storage for both the collector and the database.

Experiments & Results

We evaluated the proposed approach using a test application that consists of a three-stage unidirectional pipeline with no branching. Communication between pods uses the default Kubernetes network and HTTP/1. In each service, processing is emulated with a sleep operation whose duration is determined by request metadata, a service-specific value, and a random value drawn from a normal distribution. This setup emulates a streaming workload where processing time scales with data size — for example, video inference where larger frames lead to longer processing times. Each service drops a request when it detects a bottleneck, causing span loss.

To obtain ground-truth data, we instrumented the same application with the OpenTelemetry APIs & SDKs, which provides accurate distributed traces through header propagation. We compared the mappings inferred by the proposed method against this ground truth.

The graph shows that accuracy is approximately 96–100% when there is no span loss, but decreases significantly as the span loss rate increases.

Future Work

• Improve mapping accuracy under high span loss rates: The main reason for the accuracy drop is that the estimated delay distribution can diverge from the true distribution under high span loss rates. To address this issue, we plan to improve the initialization of the distribution and the update strategy.
• Support more complex applications: Applications with branches, where one span maps to multiple spans, require multi-dimensional assignment, so linear sum assignment is not sufficient.
• Integrate the method with OpenTelemetry or similar frameworks: This integration would automate the collection process and allow our method to be used with other observability tools.

[KubeCon Europe 2026] Reimagining Auto-Instrumentation: Prototyping Distributed Tracing Without… was originally published in nttlabs on Medium, where people are continuing the conversation by highlighting and responding to this story.

In recent years, browsers have become powerful enough to run many rich applications, such as video, games, 3D rendering, and even LLMs.

At the same time, access to system resources is heavily restricted for security reasons. Because of this, many native applications cannot run directly in the browser.

To address this limitation, projects such as v86 and container2wasm run CPU emulators on top of WebAssembly and boot a guest OS. This makes it possible to run almost any native application in the browser. However, since these approaches rely on CPU emulation, they suffer from significant performance overhead.

elfconv v1.0.0 Release

elfconv converts Linux applications into WebAssembly ahead of time (AOT), allowing them to run much faster than emulator-based solutions.

elfconv enables several use cases:

1. Running Linux applications that rely heavily on Linux features (e.g., shells or compilers) directly in the browser
2. Moving server-side Linux applications to the client, improving privacy and reducing server load
3. Building browser applications using existing Linux libraries distributed via tools like apt

There are already several projects that port native applications to WebAssembly, such as Pyodide, PGLite, and ffmpeg.wasm. However, these ports usually require significant manual effort, and keeping up with upstream changes can also be challenging. elfconv has the potential to make this process much easier.

For more details about performance and optimization techniques, please refer to our previous blog post: elfconv: Linux Apps to High-Performance Wasm Binary Translator

After continuous improvements, we are happy to announce the release of elfconv v1.0.0! 🚀

In this post, we introduce the main updates in this version.

1. Enhanced demos

In this release, we added browser demos that run GNU Bash and Python. These demos run Linux applications built from upstream source code without any modifications.

This shows that elfconv is moving beyond toy programs and benchmarks, and is becoming capable of running real-world Linux applications.

2. Multi-process support in the browser

We implemented support for clone and execve system calls. This makes it possible to run multi-process programs in the browser, including GNU Bash as shown in the demos.

Technically, each Linux process is mapped to a Web Worker. A lightweight runtime called js-kernel acts like an operating system, handling file systems and process management. Each Web Worker communicates only with js-kernel.

In addition, Wasm memory for each worker is managed by js-kernel, and resources are isolated across workers. This design provides resource isolation similar to Linux processes.

Overall, this architecture closely resembles the Linux process model.

3. Host file system mounting

Many native applications depend on external data and files. To support this, we added a feature to mount the host file system into the browser’s MEMFS. This feature is based on emscripten --preload-file function.

By setting environment variables MOUNT_SETTINGat build time, specific directories can be mounted and accessed by the application. We can generate the Wasm with mounting the host file system as follows.

outdir/$ TARGET=aarch64-wasm INITWASM=1 MOUNT_SETTING="/host/dir@/mount/point" ./elfconv.sh /path/to/elf

4. Improved Linux syscall support

In elfconv, Linux system calls for browsers are implemented in as a large switch statement.

Up to v1.0.0, we added and improved many syscalls (e.g., ppoll, ioctl, pipe2). As a result, commands such as ps, vi, and top now work in busybox.

5. Build system overhaul

The elfconv build system is based on remill. Previously, remill only supported limited environments (ubuntu 22.04 + LLVM 16) and depended on cxx-common, which had not been maintained for over two years.

With the introduction of a new superbuild system by Duncan Ogilvie, support for many Linux platforms has improved, and the dependency on cxx-common has been reduced.

We also improved the container-based build environment, significantly reducing build time and image size. As a result, the build process is now much more stable.

Future work

We are mainly working on supporting Linux shared objects. Once implemented, this will allow many pre-built binaries distributed through package managers to be ported directly to WebAssembly.

We also welcome any issues or pull requests to elfconv.

we at NTT are looking for people interested in working with us in open source communities, especially in areas such as WebAssembly and systems software. Please visit <https://www.rd.ntt/e/sic/recruit/> for more information!

私たちNTTは、WebAssemblyやシステムソフトウェアの分野において、オープンソースコミュニティと共に活動する仲間を募集しています。ぜひ弊社の採用情報ページ (https://www.rd.ntt/sic/recruit/) をご覧ください。

elfconv v1.0.0: Real-world Linux applications running fast in your browser was originally published in nttlabs on Medium, where people are continuing the conversation by highlighting and responding to this story.

In other words, gomodjail provides a “container” engine for Go modules but with finer granularity than Docker containers, FreeBSD jails, etc.

gomodjail focuses on simplicity; a security policy for gomodjail can be applied just by adding // gomodjail:confined comment to the go.mod file of the target program, and running it with thegomodjail run command.

Background: open source is under attack

Software is practically never written from scratch; It’s always assembled from enormous library dependencies, often including open-source ones. This supply chain is under attack:

• xz/liblzma backdoor incident (2024): A backdoor was injected to xz/liblzma by its maintainer (not by the original author), who had been making harmless contributions to the project for more than two years. This incident proved that even maintainers of widely adopted libraries cannot be blindly trusted.
• Massive campaign of fake Go modules (circa 2025-): In spring 2025, hundreds of fake Go modules were published on GitHub. The repositories impersonated genuine ones but contained malicious code. In some cases, the numbers of the GitHub stars even exceeded those of the genuine repositories.
• Slopsquatting (circa 2024-): AI coding agents may hallucinate and inject malicious dependencies with plausible package names. Even when an LLM itself doesn’t hallucinate, it can be still deceived by fake sites on the Internet. The chat session below shows Microsoft Copilot being deceived to suggest downloading a malicious copy of 7-zip from a fake “official” site.

Library sandboxing

Library sandboxing means confining capabilities of a library so that it cannot perform specific operations, such as reading/writing arbitrary files and executing arbitrary shell commands. This is similar to containers such as Docker and FreeBSD jails, but it differs from containers in that the confinement applies at the granularity of a library, not an OS process.

For example, Firefox adopted RLBox in 2021 to wrap C library calls in a WebAssembly sandbox. However, library sandboxing hasn’t seen wide adoption, perhaps due to its complexity: sandboxing a library typically takes days, not minutes.

“On average, sandboxing a library takes only a few days”
 — https://www.usenix.org/system/files/sec20_slides_narayan.pdf

Introducing gomodjail

gomodjail is a library sandboxing tool for Go, focusing on simplicity.

Take a look at examples/victim/main.go :

package main

import (
 "fmt"

 p "github.com/AkihiroSuda/gomodjail/examples/poisoned"
)

func main() {
 const x, y = 42, 43
 fmt.Printf("%d + %d = %d\n", x, y, p.Add(x, y))
}

The code is expected to just print 42 + 43 = 85 without any side effects. However, the Add(x, y) function here is poisoned to execute a “malicious” command:

$ go build
$ ./victim
*** ARBITRARY SHELL CODE EXECUTION ***

This 'vi' command was executed by the 'github.com/AkihiroSuda/gomodjail/examples/poisoned' module.

This example is harmless, of course, but suppose that this was a malicious code.

Type ':q!' to leave this screen.

gomodjail can confine this poisoned module so that it cannot execute such commands. It can be applied in just the following two steps:

• Step 1: Make sure that go.mod has the comment directive //gomodjail:confined

require github.com/AkihiroSuda/gomodjail/examples/poisoned v0.0.0-00010101000000-000000000000 // gomodjail:confined

• Step 2: Run the program with the gomodjail run command:

$ gomodjail run --go-mod=go.mod -- ./victim
level=WARN msg=***Blocked*** syscall=pidfd_open module=github.com/AkihiroSuda/gomodjail/examples/poisoned

How it works

gomodjail hooks dangerous syscalls such as open() and execve() using seccomp on Linux, or DYLD_INSERT_LIBRARIES on macOS. When a hooked syscall is executed, gomodjail unwinds the call stack to identify the Go module that invoked the syscall. If the module belongs to a blocklist, gomodjail blocks the syscall and injects EPERM as errno .

This run-time approach comes with several caveats; notably it is not applicable to modules that import unsafe , reflect , C, etc., since such modules may alter the call stack. A future version of gomodjail may incorporate a compilation-time approach to mitigate these caveats.

Meet me at FOSDEM 2026 for further details

I’ll talk about gomodjail at FOSDEM:

• Title: “gomodjail: library sandboxing for Go modules”
• DevRoom: Go (UB5.132, Building U)
• Date: February 1, 2026 (Sunday)
• Time: 12:00–12:30

Feel free to visit my session for further details about the project.

Besides that, I’ll also have a session titled “Lima v2.0: expanding the focus to hardening AI” on Saturday (Jan 31, 15:30–16:00). Lima has been an adopter of gomodjail, although gomodjail is not going to be the main topic in the session on Saturday.

NTT is hiring!

We at NTT are looking for engineers who work in Open Source communities in the fields of software supply chain security, sandboxing, etc. Visit <https://www.rd.ntt/e/sic/recruit/> to see how to join us.

私たちNTTは、ソフトウェアサプライチェーンセキュリティやサンドボックスなどの領域のオープンソースコミュニティで、共に活動する仲間を募集しています。ぜひ弊社採用情報ページをご覧ください: <https://www.rd.ntt/sic/recruit/>

gomodjail: library sandboxing for Go modules was originally published in nttlabs on Medium, where people are continuing the conversation by highlighting and responding to this story.

こんにちは、NTTソフトウェアイノベーションセンタ システムソフトウェアプロジェクト（SIC SOG）の田仲です。こちらは、Nemo RL で Agentic-RL を試してみた、という記事になります。

Agentic-RL とは

まず最初に、Agentic-RL についてご説明します。Agentic RL（Agentic Reinforcement Learning）は、「目標に向けて自分で考えて行動するエージェント」を強化学習で作る、機械学習手法の一つです [1]。 エージェントは観察→計画→行動→評価を繰り返し、報酬（うまくいった度合い）を手がかりに行動方針を少しずつ改善（モデル更新）します。長い手順のタスクで「次に何をすべきか」を自律的に判断する点が特徴で、過去の強化学習のようなゲームを解くだけでなく、ツール操作や作業の自動化など、現実の複雑な課題に応用が効きやすいです。

実環境において、事前学習済みモデルをそのままデプロイして満足なケースはまれで、作業状況や道具、タスクに対してエージェントを先鋭化・適応させていく継続的な学習が必要になります [2]。新しい知識や手順の追加は、SFT（教師あり微調整）で学ばせることができます。一方で、状況に応じた行動の選び方をより安定させるには、RLを用いるのがトレンドになっております [3]。つまり、データを収集しながら継続的にSFT・RLを実施することで、現場に適応したAIエージェントが確立できると考えられております。

Agentic-RL を実践するには

現在、Agentic-RL を実践するためのフレームワークは多数存在しております。興味ある方は、https://github.com/thinkwee/AgentsMeetRL、などが参考になると思います。SIC SOG では現在、Microsoft 社製の Agent Lightning、Nvidia 社製の Nemo RL、について知見を蓄積中でございます。

Agentic-RL やってみよう

本章では練習問題として、LangGraph の Self-RAG（自己評価付き RAG）エージェントを題材に、NeMo RL の GRPO（Group Relative Policy Optimization）による強化学習を適用し、「ステップ数を減らしつつ品質と根拠整合性を保つ」Agentic-RL フレームワークを簡単に組み立てます。なお、このブログでは各種手法の詳細については触れないため、正確な情報は各参考文献を参照してください。

Self-RAG とは
RAG（Retrieval-Augmented Generation）[4] は外部知識を参照して回答を生成する定番手法ですが、実運用では「検索が本当に必要か」を問わずに一定回数の検索を挟んでしまい、レイテンシとコストがかさむ要因となります。さらに、取得した文書が外れていても生成自体は進むため、回答が根拠に十分支えられないままそれらしく書けてしまう問題が残ります。Self-RAG [5] は、この課題を「モデル自身が検索と生成を自己評価しながら進める」ことで解決した枠組みです。具体的には、必要なときだけ検索するオンデマンド retrieval を可能にし、場合によっては検索を完全にスキップしたり、生成途中に追加で検索したりできます。この Self-RAG を LangGraph のノード分割と評価器で実装したのが、LangGraph Self-RAG になります（詳細は後述）。

モチベーション
しかしながら、Self-RAG を実運用してみると「検索の要/不要の判断が甘く、結局毎回検索する」「根拠と回答の整合性をどう測るか」が目立ってきます。これらはモデルの知識不足というより、いつ検索し、どの根拠で答えるかというエージェントの自律性・行動方針の問題です。そこで、エージェントの制御そのものを学習で改善する Agentic-RL が必要になります。

ゴールと評価指標
このプロジェクトのゴールは「最終回答までのステップ数を減らしつつ、回答品質と根拠整合性を保つ」とします。ステップ数は、過剰な検索やループを抑える直接のターゲットになります。品質は、質問に答えているかという観点の評価で、根拠整合性は回答がコンテキストに裏付けられているかという観点の評価です。

エージェントの状態 AgentState を設計する
前述の指標をエージェント内部状態に落とし込めるよう設計します。LangGraph ではノード間で状態を受け渡すために、全ノードが共通で参照できる AgentState を定義します。今回は、AgentState に 会話履歴、正規化された質問、取得した文書、最終回答、各種カウンタ（rewrite_count や tool_call_count）を入れました。

class AgentState(TypedDict, total=False):
    messages: Sequence[BaseMessage]           # Human/AI/ToolMessage の履歴
    question: str | None                      # 正規化された質問
    documents: List[str] | None               # RAG 生成に使うコンテキスト
    generation: str | None                    # 最終回答
    rewrite_count: int                        # リライト回数
    tool_call_count: int                      # ツール呼び出し回数
    placeholder_failures: int                 # 曖昧な最終回答の回数
    sp: List[List[Any]] | None                # Supporting Facts
    sp_structured: List[Dict[str, Any]] | None
    metadata: Dict[str, Any]                  # ground truth, provenance など

rewrite_count や tool_call_countを状態として持つことで、エージェントのメタ情報をNemo RL に渡すことができ、ループ抑制や報酬設計がかのうになります。

agent / retrieve / rewrite / generate の制御フローを作る
Self-RAG のフローを以下の図に示します。コアは 4 つのノードと1つのエッジで、agent ノードは「検索するか、最終回答に行くか」を決め、retrieve_node が実際の検索を実行します。 workflow.add_conditional_edges("retrieve", grade_result) が「この文脈で生成して良いか、それとも質問をリライトすべきか」を決め、必要なら rewrite ノードにて質問を書き換えて agent ノードに戻します。十分な文脈があると判断したら generate ノードにて回答を生成して終了します。

workflow.add_conditional_edges("retrieve", grade_result) は、生成モデル自身を評価器として使い、判断結果grade_resultの数値に落とします。ここでgrade_resultは 、GradeDocuments は取得文書が質問に十分関連しているか、GradeDocuments は回答がコンテキストに裏付けられているか、の判定結果です。

環境と報酬モデルを作る
RL では「1 回の実行」を 1 エピソードとして扱うため、LangGraph を 1 回走らせてメトリクスを集計します。ここで集めたいのは、ステップ数、検索回数、リライト回数、最終回答、取得文書、そして ground truth（正解）に基づく Exact Match（EM）や F1 などです。次に、これらのメトリクスと LLM grader のスコアを合成して報酬を決定します。報酬は「速さ」「品質」「根拠」などのモデルの伸ばしたい能力を決める設計図です。今回は、以下のように設計しました。

【ツール使用回数・リライト回数の削減を狙った報酬】

• 多いと負の報酬を与えるようにした。当初、正の報酬を与えていたら、低精度・低ステップでの回答をするようになってしまい負の報酬とした。（所謂、報酬ハッキング[]というやつを実際に見れて感動した。）
• 報酬ハッキングを起こさない工夫として、閾値以下ならば追加で正の報酬を与える posterior GRPO を使用した。

【正解を用いた引用文章の正確さ・F1スコアの向上を狙った報酬】

• グランドトゥルースを用いて評価し、正負の報酬を与える
• どちらも個別に報酬を与え合計している。その際に、望ましくない以下のようなパターンで学習しないように、以下のような場合には報酬の合計値がゼロになるように設定した。
  * 引用が間違っている（負の報酬）が、F1スコアが高い（正の報酬）
  * 引用は正しい（正の報酬）が。F1スコアが低い（負の報酬）

LangGraph と Nemo RL をブリッジする
NeMo RL 側では主に「message_log_batch」と「metadata」を入力として扱います。一方、Self-RAG（LangGraph）側の評価では、取得した文書や通過したノードなどの詳細な実行ログが得られます。そこで、Self-RAG の実行中に得られた情報のうち報酬計算に必要なもの（ground truth、取得文書、通過ノードに加えて、取得回数・再検索回数・ノード通過数などの各種カウンター）を metadata にまとめて付与し、NeMo RL に渡します。報酬計算時には message_log_batch から質問・応答を取り出し、metadata の付帯情報と突き合わせて EpisodeMetrics を組み立てます。最後に、EpisodeMetrics に格納された取得結果・経路・カウンター値を用いて報酬テンソルを算出し、NeMo RL に返します。

GRPO
GRPO[7] は、同一の質問（prompt）に対して複数の候補応答を同時に生成し、それらを同一質問内で相対比較することで方針を更新する。具体的には、1 回の学習ステップで複数の prompt をまとめて扱い、各 prompt につき複数の応答候補をサンプリングする。候補ごとに得られた報酬は prompt 単位でグループ化され、同一グループ内の平均との差分（必要に応じて分散で正規化）から advantage を構成する。この advantage を用いて、良い候補応答の生成確率を上げ、相対的に悪い候補の確率を下げる方向に、PPO 系の clipped policy gradient と KL 正則化を組み合わせて重みを更新する。

評価

評価環境

• 使用したモデル：Qwen2.5–1.5B-Instruct [8]
• 使用したGPU：Nvidia A100 x2
  LangGraph 用に1台、Nemo RL 用に1台、という構成です。
• 使用したデータセット
  - Hotpot QA[9]：学習と評価に使いました。
  - Fever[10]：強化学習の汎化能力を観察する意図で、学習には用いず、評価にのみ使用しました。
• 学習の設定
  先行の知見として、学習初期ステップでは簡単な問題を用い、学習が進んだタイミングで難しい問題に切り替えるスケジューリングの有効性が知られており[11]、本実験では Hotpot QA の 難易度 タグに基づき、2つの難易度に学習データを分割し2段階の強化学習を施しました。

評価結果（Hotpot QA）
評価として、事後学習なし（Baseline）、Hotpot QA の easy のみで事後学習、 easy と medium の混在で事後学習、の 3 条件のSelf-RAG ループ数（agent → retrive → rewrite を1ループとした）と F1スコアで比較しました。

平均ループ数の大幅な削減、F1スコアの微増が確認されました。平均2.42ループであったが、検索なしで直接回答した件数は Baseline から 事後学習後で 140% 増大しており、”不要なRAG検索をしない” という目的が達成できていることを確認しました。また、目的達成のために精度を犠牲にしていない点も確認できました。

学習挙動（エントロピーと報酬）
エージェントのエントロピー（ランダム性が高いほどエントロピーが高い）[12]が、強化学習ステップと共に減少していることがわかりました。よって、強化学習によりエージェントの意図していない動作を抑制できる可能性が示されました。

エージェントが獲得した報酬も、強化学習ステップと共に増大しており、強化学習がスムーズに進んでいることがわかりました。また、今回は 18 step までの学習としているが、報酬がサチっていないことから、学習が飽和したというより、まだ改善余地が残っているように見えています。

評価結果（Fever）
学習には用いていない Fever データを用いて評価した結果、こちらでもループ数の削減と、精度の微増が確認された。Fever は Hotpot QA とタスク形式が異なるため、この改善は特定データセットへの過適合というより、検索・停止・リライトの意思決定というメタな振る舞いが学習された可能性を示します。

感想、示唆

報酬設計
今回、報酬の設計に一番苦労しました。報酬は「何を良しとするか」を数値に落とす作業ですが、少しの重み変更で方策が別方向へ収束します。本実験でも、ツール使用回数やリライト回数に素直に正のインセンティブを置いた際に、即回答でも根拠が弱い回答を量産するなど、報酬ハッキングが起きました。有効な報酬設計には、エージェントにどんな振る舞いをしてほしいか、に関する正しい定義・理解で求められるように思いました。

低い計算コスト
今回の設定は 12時間/GPU 程度で学習が回り、試行錯誤を回すことができました。Agentic-RL は設計空間が広く、まずは「短いサイクルで壊して直す」ことが重要になります。学習が重いと報酬設計の探索が困難になるため、この計算規模で動いたこと自体が実務上有益です。また、システムとしての最適化余地は大いにあると考えており、弊研究グループの力の見せ所だと考えております。

今後の展望
本ブログは、最高性能の追求というより「LangGraph にてエージェントを定義し、報酬を返し、GRPO で更新できる」ことの検証がメインです。LangGraph 側の状態遷移とログを、NeMo RL が扱えるメッセージログとメタデータに落とす橋渡しが理解できたことで、同じ枠組みを他のエージェントにも適用できるのではないか、と考えております。今後、この技術を横展開することによって、社内外のAIエージェントに対しても評価・改善のループを回し、AIエージェントの運用コストの削減と挙動の安定化を狙います。

SIC SOG では、本日紹介した、ワークフロー設計、Agentic-RL の実装、以外にも、プロンプトの自動改善、SFT による知識注入、エージェントのメトリクス収集、についても知見を蓄積し、実運用で使えるエージェント継続学習基盤の確立を目指しています。AIエージェントの開発は、研究とプロダクトの接続点そのものです。エージェントの評価設計、報酬設計、学習基盤、運用までを一気通貫で扱いたい方・ご興味ある方は是非ご連絡ください。

参考文献

[1] Zhang, G., et. al., “The Landscape of Agentic Reinforcement Learning for LLMs: A Survey,” https://arxiv.org/abs/2509.02547.
[2] Ouyang, L., et. al., “Training language models to follow instructions with human feedback,” https://arxiv.org/abs/2203.02155.
[3] Liu, M., et. al., “ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models,” https://arxiv.org/abs/2505.24864v1.
[4] Lewis, P., et. al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” https://arxiv.org/abs/2005.11401.
[5] Asai, A., et. al., “Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection,” https://arxiv.org/abs/2310.11511.
[6] Skalse, Joar., et. al., “Defining and Characterizing Reward Hacking,” https://arxiv.org/pdf/2209.13085.
[7] Shao, Z., et. al., “DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models,” https://arxiv.org/abs/2402.03300.
[8] Qwen Team, “Qwen2.5 Technical Report,” https://arxiv.org/abs/2412.15115.
[9] Yang, Z., et. al., “HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering,” https://arxiv.org/abs/1809.09600.
[10] Thorne, J., “FEVER: a large-scale dataset for Fact Extraction and VERification,” https://arxiv.org/abs/1803.05355.
[11] Narvekar, S., “Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey,” https://arxiv.org/abs/2003.04960.
[12] Ahmed, Z., “Understanding the impact of entropy on policy optimization,” https://arxiv.org/abs/1811.11214.

Agentic RL in Practice: Optimizing a LangGraph Self-RAG Agent with NeMo RL (GRPO) was originally published in nttlabs on Medium, where people are continuing the conversation by highlighting and responding to this story.

LinuxカーネルのRust製モジュールが普及するのはまだ先だから、厳しく精査されないだろうと安心して適当な記事を書いてきました。ところが、今年6月にリリースされたAndroid 16には、Rustで実装されたカーネルモジュールが含まれており、すでに広く普及していました。

今回は、カーネル用に独自実装されたRustのアトミック型について紹介します。

メモリモデル

Linuxカーネルは、Rustの標準ライブラリのアトミック型を使わず、独自のアトミック型を実装しています。その大きな理由は、Rustの標準のアトミック型が前提とするメモリモデルとカーネルのメモリモデルLKMM（Linux kernel Memory Model）が異なるからです。

「メモリモデル？」という方も読み進められる技術レベルのことしか書かれてないので、安心してください。メモリモデルは、複数CPU（複数スレッド）から見たときに、メモリの読み書きがどの順序で観測されうるか、そして同期操作が何を保証するかを定めたルールだと考えてください。命令はコンパイラ最適化やCPUの再順序化があるため、直感どおりに「上から実行される」とは限りません。

メモリモデルの前提が揃っていないと、例えばRustとCが同じメモリ領域に並行してアクセスする場合に、読み書きが観測される順序や可視性についての理解が食い違い、開発者が意図した同期が成立しない可能性があります。

Rust自体にLKMMサポートを加える提案もありましたが、最終的にはLinuxカーネル側で独自のアトミック型を実装する設計に落ち着きました。

汎用アトミック型

標準ライブラリのアトミック型は、AtomicI32やAtomicI64のように、プリミティブ型ごとに専用の型が用意されています（AtomicBoolやAtomicUsizeなども同様）。一方、カーネル用のアトミック型は Atomic<T>という型パラメータ付きの形で定義されています。

「お、標準ライブラリよりも使いやすい。いろいろな型をアトミックに？」と思った方もいるかもしれませんが、カーネル用APIがアプリケーション用APIよりも使いやすいなどということはありえません。Atomic<T>は「任意の型をアトミックにできる」わけではありません。Tは、内部表現としてi32もしくはi64に対応付けられる型に限られます。例えばisize（ビット幅はアーキテクチャ依存）や、#[repr(i32)]か#[repr(i64)]を付けたenumなどが対象です。

これは、カーネルのC側で長年使われてきた主要な整数向けのアトミックな型が32ビットのatomic_tと64ビットのatomic64_tであり、Rust側の実装もそれら向けに用意されたCのAPIを薄くラップして提供しているためです。

C側のアトミックAPIは性能を優先して多くが インライン関数やマクロとして提供されています。そのためRust側から利用する場合、バインディングの都合で呼び出しがラッパー経由になり、純粋な性能面では不利になる可能性があります。

それでもこの設計が選ばれているのは、速度面の最適化よりも、カーネルが長年使い続けてきた同期機構のセマンティクスにそのまま乗れることを重視しているためです。Rust側で同等の命令列を自前で実装しようとすると、アーキテクチャ固有命令や各種バリア、周辺機構との整合まで含めて、カーネル固有の前提を継続的に再現し続ける必要が出てきます。

その結果、性能よりも「Cとの一貫性を保ちやすい」こと、そしてコードを保守しやすいことを重視して、CのアトミックAPIへ1対1でマッピングする方針になっています。

独自アトミック型のAPIは、標準ライブラリのアトミックAPIと似せてあり、違和感なく使えるはずです。

i8もアトミックに

Linuxカーネルの開発でRustを使い始めた、という理由もなく困難な道を選んできた人を除けば、標準ライブラリで使っていたi8 やi16 のアトミック型も同じ感覚で使いたい、と考えるかもしれません。そのような希望（未確認）に対応すべく、筆者はこの点の改善に取り組んでいるため、ここでは課題を整理します。

上で説明したように、カーネルの Atomic<T>はCのアトミックAPIに1対1でマッピングする方針で設計されています。しかしC側には、i8 や i16に対応するアトミック型やAPIは用意されていません。

アトミック操作は専用のCPU命令に依存するものが多く、アーキテクチャごとに実装や検証のコストが増えがちです。Cと一貫性のあるRustのアトミック実装を各アーキテクチャで維持する負担を考えると、Rust側で独自に再実装するアプローチは関係者の合意を得にくく、C側の既存実装をできるだけ活用する方針のほうが現実的です。

まとめ

Linuxカーネル内のRustでは、標準ライブラリのアトミック型をそのまま使うのではなく、LKMMを前提にしたカーネル独自のアトミック型が採用されています。さらに、Arcのように内部で標準のアトミック型を使う型も、Cのアトミック型を使う独自実装へ置き換えられています。

同じ流れとして、時間関係の型もカーネル独自の実装に置き換えられたことも紹介しました。標準のRustライブラリからカーネルの前提で正しく動く独自ライブラリへの置き換えが進み、Rustでドライバを実装するための現実的な環境が整ってきています。

NTT研究所では、低レイヤの開発に興味がある仲間を募集中です。連絡お待ちしています。

LinuxカーネルのRustアトミック型 was originally published in nttlabs on Medium, where people are continuing the conversation by highlighting and responding to this story.

タイマーAPI基礎知識

特定の時点を時間を表すCの型としてktime_tがあり、これは64ビット符号付き整数（s64）で表現され、単位はナノ秒です。ktime_t型は期間を表す型としても使われます。

Linuxカーネルには、下記のように複数のクロックソースが用意されていて、デバイスドライバは用途や特性に応じてクロックソースを使い分けます。例えば、realtimeクロックは補正によって時間が過去に戻る場合があるので、経過時間の測定用途には不向きです。また、独自のクロックソースを実装しているデバイスドライバもあります。

経過時間の計算に関するバグ

今回は、経過時間の計算に関するバグを例に取り上げます。

経過時間を求めるときには、2つのktime_tの差を計算します。しかし、このとき異なるクロックソース由来のktime_tを比較すると不正な値になってしまう、というバグが起こり得ます。

本来であれば、必ず同じクロックソースから取得したktime_t同士で演算を行う必要があります。ところがC言語のAPIでは、ktime_t型は単なる s64として定義されているため、クロックソースが異なっていてもコンパイルは通ってしまいます。

異なるクロックソースのktime_tを誤って演算できないようにするには、ktime_tとクロックソースの関係を保持する仕組み が必要です。
C言語で考えると、コンパイル時にチェックする方法としてはクロックソースごとに独自のktime_t型を定義する ことが考えられます。技術的には可能ですが、その場合はktime_t型を使う多数のカーネルAPIをクロックソースごとに用意する 必要があり、現実的ではなさそうです。一方、実行時チェックでよいのであれば、ktime_t型にクロックソースの情報を持たせて比較する、といった方法も考えられますね。

どちらの方法も今すぐ忘れましょう。そんな実験は自作OSでやってください。

素直なRust版ktime_tの設計

Rustの型システムを使って、コンパイル時にバグを防ぐことのできる設計を考えてみましょう。

まずは、C言語のktime_tを素直にラップした構造体の一例です。

特定の時点を表す型をInstant、期間を表す型をDeltaと分けて定義しましたが、内部的にはCのktime_tを保持しています。この設計では、Instant型とクロックソースが関係付けられていないので、Cと同様に異なるクロックソース由来のInstantの演算を防ぐことはできません。Rustで単純にCをラップしただけでは、Rustのなんとなく安全な雰囲気は出せても、具体的なメリットは得られないことがわかりました。

改良したRust版ktime_tの設計

Linuxカーネルで実際に使われているRust版のInstant型は、ジェネリック型パラメータを持つ構造体として定義されています。ClockSourceは、パラメータとして使える型を制限するために用いられており、Rustでトレイトと呼ばれるものです。この記事はRustが使えるようになるためのものではないので、なんとなくわかった気になって読み進めてください。

Instant型にはClockSourceトレイトを実装した型が埋め込まれているように見えますが、実際には コンパイル時にInstant型とクロックソースを関連づけて型チェックを行うためだけに使われます。そのため、Instant型のメモリサイズはパラメータを持たない素直な設計の構造体と変わりません。また、ClockSourceトレイトを実装する型、Monotonicなど全ての構造体は空で、実行時にメモリは確保されません。

異なるクロックソースに関連づけられたInstant型は、それぞれ別の型として扱われるため、両者を演算しようとするとコンパイルエラーになります。

関数もジェネリック型パラメータをサポートしているため、どのクロックソースに関連づけられたInstant型でも引数や戻り値として利用できます。同じような関数をクロックソースの数だけ実装する必要はありません。

まとめ

Rustの型システムを活用することで、CのAPIをラップした、より安全なAPIを実現できそうですね。

NTT研究所では、低レイヤの開発に興味がある仲間を募集中です。連絡お待ちしています。

Rustの型システムでLinuxカーネルのバグを防ぐ was originally published in nttlabs on Medium, where people are continuing the conversation by highlighting and responding to this story.

Software supply chain under attack

Homebrew is the most popular open source package manager on macOS, with more than 7700 package formulae. The large number of packages is both a strength and a weakness; While having many packages is certainly convenient, it raises doubts about whether the entire source code for all of them has been comprehensively reviewed.

For example, the notorious xz/liblzma backdoor incident (CVE-2024–3094) has shown even well-known packages can be compromised, although Homebrew was not affected in this case by chance.

This kind of supply chain attack may actually happen with any package manager; this month (July 2025) saw a very sophisticated phishing campaign that compromised several packages on npm.

AI agents may make mistakes

For better or worse, it is becoming common practice to allow AI agents to run arbitrary shell commands. This practice is extremely dangerous of course; an AI agent may hallucinate, or may be deceived by a web search result, to install malware with plausible package names:

pip install <PLAUSIBLE_PACKAGE_NAME>

To alleviate the risk of running such arbitrary commands, AI agents such as OpenAI Codex CLI and Google Gemini CLI utilize Apple’s sandbox-execcommand on macOS.

sandbox-exec -f PROFILE COMMAND [ARGS]

sandbox-exec supports limiting file access with a profile like:

(allow file-read*)
(deny file-write*)
(allow file-write* (literal “/dev/null”))

However, sandbox-exec seems to have been deprecated since circa 2016.

$ man sandbox-exec
[…]
DESCRIPTION
 The sandbox-exec command is DEPRECATED.
 Developers who wish to sandbox an app should instead adopt
 the App Sandbox feature described in the App Sandbox Design Guide.

In the manual page, Apple recommends using “App Sandbox” instead, however, App Sandbox doesn’t actually provide the direct replacement for the sandbox-exec command.

Introducing Alcoholless

Alcoholless provides a simple CLI to run shell commands with reduced security risk:

cd ~/SOME_DIRECTORY
alcless brew install xz
alcless xz SOME_FILE

In the example above, xz works as a separate user with an access for the copy of the current directory. Changed files are synced back to the current directory when the command exits.

How it works

Alcoholless just utilizes 1990s’ commands ( su , sudo , rsync) and the macOS equivalent of useradd to implement container-like environments, without extending the XNU kernel to support Linux-style container syscalls. A fun fact is that both su and sudo have to be utilized because sudo can’t fully switch the user context on macOS by itself, due to Mach’s quirks that are not recognized by POSIX.

Alcoholless could even harden security if it utilized Apple’s Virtualization.framework, however, it doesn’t do that, as the framework apparently does not provide a way to automate the initialization steps of a macOS VM (accept EULA, skip enabling iCloud, set up SSH, etc.).

This barrier is annoying, but it also comes with several bonuses in avoiding virtualization:

• No performance overhead
• Minimal disk consumption
• Direct access to the host hardware (GPU, etc.)
• Works fine on GitHub Actions (no nested virtualization)

Getting started

Alcoholless can be installed from the source code as follows:

brew install go
git clone https://github.com/AkihiroSuda/alcless.git
cd alcless
git checkout v0.1.1
make
sudo make install

Alternatively, you can also download binary packages from <https://github.com/AkihiroSuda/alcless/releases>.

For the first run, the alclessctl create default command has to be executed. You’ll be asked to type the password to create the new user account alcless_${USER}_default :

$ alclessctl create default
7:41PM INF Creating an instance instance=default instUser=alcless_user_default
⚠️  The following commands will be executed:
sudo sysadminctl -addUser alcless_user_default -password -
sudo chmod go-rx /Users/alcless_user_default
sudo sh -c 'echo '"'"'suda ALL=(root) NOPASSWD: /usr/bin/su - alcless_user_default -c *'"'"' >'"'"'/etc/sudoers.d/alcless_user_default'"'"''
❓ Press return to continue, or Ctrl-C to abort
[RETURN]
CONTINUE
7:42PM INF Running command cmd="sudo sysadminctl -addUser alcless_user_default -password -"
2025-07-21 19:42:06.758 sysadminctl[37537:5738895] ----------------------------
2025-07-21 19:42:06.758 sysadminctl[37537:5738895] No clear text password or interactive option was specified (adduser, change/reset password will not allow user to use FDE) !
2025-07-21 19:42:06.758 sysadminctl[37537:5738895] ----------------------------
User password: [PASSWORD]
[...]

You may also have to verify that your home directory has restrictive permissions:

$ ls -ld ~
drwxr-x---+ 55 user staff  1760  7 22 05:19 /Users/user/

$ chmod 700 ~

$ ls -ld ~
drwx------+ 55 user staff  1760  7 22 05:19 /Users/user/

Basic usage

After the completion of alcless create default, you have to cd to a directory that is to be synced to the alcess_${USER}_default environment:

mkdir -p ~/tmp
cd ~/tmp
# Create some content in the current project directory
echo foo >foo

Then you can install and run a Homebrew package such as xz :

$ alcless brew install xz
[...]

$ alcless xz foo
7:44PM INF ➡️Syncing the files src=/Users/user/tmp/ dst=default:/Users/alcless_user_default/Users/user/tmp
7:44PM INF ⬅️Syncing the files back (dry run) src=default:/Users/alcless_user_default/Users/user/tmp/ dst=/Users/user/tmp
*deleting foo
.d..t.... ./
>f+++++++ foo.xz
7:44PM INF ⬅️Syncing the files back src=default:/Users/alcless_user_default/Users/user/tmp/ dst=/Users/user/tmp
⚠️  The following commands will be executed:
rsync -rai --delete -e '/usr/local/bin/alclessctl shell --workdir=/ --plain' default:/Users/alcless_user_default/Users/user/tmp/ /Users/user/tmp
❓ Press return to continue, or Ctrl-C to abort
[RETURN]
CONTINUE
*deleting foo
.d..t.... ./
>f+++++++ foo.xz

alcless syncs the current directory /Users/${USER}/tmp to /Users/alcless_${USER}_default/Users/${USER}/tmp , executes the specified command using the user credential of alcless_${USER}_default, and syncs back the directory with a confirmation prompt.

Usage with Gemini

Gemini CLI can be installed in Alcoholless as follows:

alcless brew install gemini-cli

This command may take around 10 minutes due to recompilation of several dependency packages such as node. This recompilation is needed as the Homebrew prefix (/Users/alcless_${USER}_default/homebrew) differs from the standard installation of Homebrew (/opt/homebrew).

Then add your GEMINI_API_KEY to .zshenv inside Alcoholless:

alcless sh -c 'vi ~/.zshenv'

Now you can run Gemini CLI and let it run arbitrary shell commands inside Alcoholless:

$ alcless gemini

╭─────────────────────────────────────────────────────────────╮
│  > Install a Python package that shows the current weather  │
╰─────────────────────────────────────────────────────────────╯

 ╭──────────────────────────────────────────────────────────────────────────────────────────────────────╮
 │ ✔  GoogleSearch Searching the web for: "python package current weather"                              │
 │                                                                                                      │
 │    Search results for "python package current weather" returned.                                     │
 ╰──────────────────────────────────────────────────────────────────────────────────────────────────────╯
✦ I will install the python-weather package, which allows you to get the current weather for a location.
 ╭─────────────────────────────────────────────────────────────────────────────────────────────────────────╮
 │ ?  Shell pip install python-weather (Install the `python-weather` package using pip.) ←                 │
 │                                                                                                         │
 │   pip install python-weather                                                                            │
 │                                                                                                         │
 │ Allow execution?                                                                                        │
 │                                                                                                         │
 │ ● 1. Yes, allow once                                                                                    │
 │   2. Yes, allow always "pip ..."                                                                        │
 │   3. No (esc)                                                                                           │
 │                                                                                                         │
 ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯

It should be noted that Alcoholless is not a panacea; it is still highly recommended to review AI-generated commands before execution.
While the AI is unlikely able to steal or falsify a file outside the working directory (unless macOS has a bug around the user isolation), it might be still able to exploit other attacks such as cryptomining or denial-of-service.

NTT is hiring!

We at NTT are looking for engineers who work in Open Source communities in the fields of software supply chain security, AI sandboxing, etc. Visit <https://www.rd.ntt/e/sic/recruit/> to see how to join us.

私たちNTTは、ソフトウェアサプライチェーンセキュリティやAIサンドボックスなどの領域のオープンソースコミュニティで、共に活動する仲間を募集しています。ぜひ弊社採用情報ページをご覧ください: <https://www.rd.ntt/sic/recruit/>

Alcoholless: A Lightweight Security Sandbox for macOS Programs (Homebrew, AI Agents, etc.) was originally published in nttlabs on Medium, where people are continuing the conversation by highlighting and responding to this story.