A HuggingFace paper presents OpenForgeRL, an open-source framework for training agents that use complex harnesses like Claude Code or Codex. Until now the problem was practical: open training stacks (SFT and RL) can’t handle the stateful, multi-process inference that a real harness requires. You could evaluate an agent in a harness, but not train it there.

OpenForgeRL solves this with two pieces. A lightweight proxy intercepts model calls during harness use and logs them as training data for a standard RL codebase (veRL). A Kubernetes orchestrator runs each rollout in a separate remote container. Training and inference are decoupled: the model learns in the real environment where it gets deployed.

As we covered on July 20, orchestration matters more than the model in production agents. OpenForgeRL brings that principle into training: the harness becomes where the agent learns, not just where it works.

Results are measured. OpenForgeGUI reaches 37.7 on OSWorld-Verified and 72.3 on WebVoyager, beating open baselines of similar size and matching models several times larger. The paper is honest about limitations: error recovery stays weak even after RL, and some harnesses are substantially harder to learn than others.

If you want to try it: the code is public, but you need a Kubernetes cluster and familiarity with RL stacks like veRL. It’s reproducible research, not an afternoon of work.

In detail

The idea of “harness” deserves clarification. When you use Claude Code or Codex, the model doesn’t work alone: a software architecture around it manages context, calls tools, decides when to stop and retry. That architecture is the harness. It’s what makes the difference between a model answering a question and an agent completing a multi-step task.

The training problem is that this architecture is stateful: it maintains state between calls, runs on separate processes, accesses filesystem and network. Open reinforcement learning stacks like veRL assume a training episode is a linear sequence of model calls. A real harness breaks that assumption: each step can spawn subprocesses, write files, make network calls, and the result of an action depends on everything that happened before in non-trivial ways.

OpenForgeRL’s solution is elegant in its simplicity. Instead of modifying the RL stack to understand the harness, it puts a proxy in the middle. The proxy acts as a go-between for model calls: the harness thinks it’s talking to the real model, the proxy logs everything and then replays the trajectory for training. On the orchestration side, each training episode runs in an isolated Kubernetes container, so the environment is clean and reproducible.

The numbers need reading in context. On ClawEval, OpenForgeClaw reaches 31.7 pass@1 and 55.9 pass@3 with hundreds or thousands of tasks. On GUI benchmarks, OpenForgeGUI gets 37.7 on OSWorld-Verified and 72.3 on WebVoyager. In the GUI case, results match or beat models several times larger, suggesting that training in the real harness matters more than model scale.

The paper reports two findings that count for those building agents. First: RL improves agent reliability, particularly self-verification, tool coverage, and multi-step plan completion. Second: error recovery stays weak. An agent trained with RL learns to use tools better when everything goes smoothly, but doesn’t learn to recover when something breaks halfway through. It’s a structural limit, not a bug to fix with more data.

The third finding concerns harnesses themselves: some are substantially harder for the model to learn regardless of training quality (ZeroClaw, OpenClaw and Codex give very different results with the same model). This confirms a point becoming solid in research: harness choice changes agent performance as much as model choice.

What not to conclude: OpenForgeRL doesn’t mean training agents got easy. You need a cluster, RL familiarity, and tasks must be well-designed. The paper uses hundreds or thousands of tasks, but each must be verifiable. For those wanting to build agents with open stacks, the value is that the gap between proprietary and open infrastructure narrows on the training front, not just inference, where open-weight models like Laguna S were already catching up.