Reinforcement learning for a 744B-parameter agent

The training loop: synthetic tasks go to the production agent harness, where the 744B mixture-of-experts policy acts; verifiers score the result as rewards, and a GRPO update improves the policy. Synthetic tasks Policy 744B mixture of experts Agent harness the production one Verifiers and rewards GRPO update on the Slime framework acts trajectories reward
The training loop. The policy acts inside the production agent harness on synthetic tasks, verifiers turn each trajectory into a reward, and a GRPO update improves the policy.

We trained a 744-billion-parameter mixture-of-experts agent end to end with reinforcement learning, using the production agent harness itself as the environment, so the model learns in the same setting where it is later used.

Main results

  • An initial small-scale run raised held-out task reward by 0.077 over the base policy.
  • Generation took 61% of wall-clock time; with FP8 weights one rollout engine fits on each node, which gave 2.6 to 4.1 times the output of the previous two-node layout.
  • Asynchronous one-step off-policy training cut per-step non-generation time from 170 seconds to 24 seconds.