Reinforcement learning for a 744B-parameter agent
We trained a 744-billion-parameter mixture-of-experts agent end to end with reinforcement learning, using the production agent harness itself as the environment, so the model learns in the same setting where it is later used.
Main results
- An initial small-scale run raised held-out task reward by 0.077 over the base policy.
- Generation took 61% of wall-clock time; with FP8 weights one rollout engine fits on each node, which gave 2.6 to 4.1 times the output of the previous two-node layout.
- Asynchronous one-step off-policy training cut per-step non-generation time from 170 seconds to 24 seconds.