NVIDIA released Molt, a training framework for agentic reinforcement learning written in native PyTorch. The goal is to keep the cost of each research iteration low.

Today, modifying an agentic training algorithm in a mainstream framework means crossing different layers: trainer, distributed backend, and rollout glue. Infrastructure work that falls on the researcher with each experiment. Molt compresses those layers into a compact codebase, clean enough that a researcher can keep it in their head, and readable enough that an AI coding assistant can read through it entirely and reason about it.

For those experimenting with agents and training them, this means less time fighting plumbing and more time testing ideas. The paper reports that, with an equivalent asynchronous protocol, Molt is statistically comparable to a Megatron-based stack: lightness doesn’t cost performance.

The thread making agentic training accessible outside proprietary labs is growing: as we covered on July 25 with OpenForgeRL, which closed the gap between proprietary harnesses and open training, Molt now pushes on code readability.

The code is open source with recipes and containers on GitHub.

If you want to try it: the repo with recipes and containers is at github.com/NVIDIA-NeMo/labs-molt.

In detail

Agentic reinforcement learning is research made of continuous experimentation: new estimators, new pipeline phases, new rollout schemes. In mainstream frameworks, each change must propagate through three layers with different logic. The trainer orchestrates training. The distributed backend handles communication between GPUs. The rollout glue collects agent experience from the environment. The more coupled the layers, the more each experiment costs, and that cost falls on the researcher each time.

What changes with Molt. The guiding principle is simple: the codebase must be small enough to fit in your head, and clean enough that an AI coding assistant can read it entirely and modify it end-to-end. It’s an architectural choice that prioritizes modifiability. The agent is treated as an ordinary program, and a single asynchronous loop handles training of multimodal policies and mixture-of-experts. One relevant detail: the system never trains on tokens the model didn’t generate, maintaining coherence between tokens, policy versions, and model semantics. This avoids a class of subtle bugs that in traditional frameworks emerge when rollout and training desynchronize.

The comparison with Megatron. Under a matched asynchronous protocol, Molt proves statistically comparable to a Megatron-based stack. The paper makes a precise claim: compactness doesn’t cost performance. For a researcher, the difference lies in the modification cycle: changing an estimator or adding a pipeline phase means touching a codebase you understand, not three coupled layers where each change has cascading effects.

The limitations. The comparison is self-referential: NVIDIA built the tool and measured performance. “Statistically comparable” means results fall within overlapping confidence intervals, not that Molt dominates. The compactness claim is qualitative: the paper doesn’t propose a formal readability metric, it relies on the codebase being “compact and clean” according to the authors. The design is intended for researchers doing agentic RL with access to multiple GPUs: if your use case is training an agent on a specific task with modest resources, Molt might be oversized. The published recipes and containers help you get started, but the learning curve remains that of someone who already knows what an asynchronous rollout is.

The code is open on GitHub with recipes and containers. For those following the thread of open-source agentic training, it’s another step toward modifiability: frameworks designed to be read and changed, not just executed.