Nvidia boosts Nemotron coding scores with targeted reinforcement learning
Nvidia used 20,000 synthetic training examples and PivotRL to improve Nemotron 3 Ultra's agentic coding capabilities, raising its Terminal-Bench score significantly.
Nvidia has significantly improved the coding performance of its open-weight AI models by targeting specific agent failures with a small, high-quality dataset. Using a specialized harness from Factory, the team generated roughly 20,000 training examples to refine Nemotron 3 Ultra, a 550-billion-parameter model released in June 2026. This focused post-training effort resulted in a notable jump in benchmark scores for agentic coding tasks.
What happened
Nvidia collaborated with Factory to address recurring issues in how Nemotron 3 Ultra handles complex coding instructions. The team deployed Factory’s Droid coding-agent harness within an air-gapped network to instrument and capture reproducible failures. These failures primarily involved instruction following, tool calling, and error recovery during multi-step tasks. By isolating these specific breakdowns, Nvidia could create a targeted dataset for model improvement rather than relying on broad, unfiltered data.
The company applied a reinforcement learning method called PivotRL to this new dataset. Introduced in March 2026, PivotRL trains models on critical decisions within existing supervised fine-tuning trajectories. This approach avoids the high computational cost of full multi-turn rollouts while still improving decision-making accuracy. Nvidia returned updated model checkpoints to Factory for evaluation after applying this technique, ensuring the improvements were measurable in real-world agent scenarios.
To prevent the model from overfitting to the specific testing environment, Nvidia varied the Droid configuration and incorporated data from several other agent environments. The final training pipeline combined reinforcement learning with verifiable rewards, human feedback, and multi-teacher on-policy distillation. This multistage process allowed capabilities from teacher models trained in different technical domains to transfer effectively into the student model, Nemotron 3 Ultra.
How it works
PivotRL differs from standard reinforcement learning by focusing on the outcome of an action rather than its similarity to a reference example. In traditional supervised fine-tuning, a model is penalized if its output does not match a golden answer exactly. PivotRL evaluates whether the action achieved the intended goal, which is crucial for agentic tasks where multiple valid paths may exist. This method achieved accuracy comparable to end-to-end reinforcement learning but required only one-quarter as many rollout turns, making it far more efficient.
The training process also utilized multi-teacher on-policy distillation. This technique transfers knowledge from several specialized teacher models into the single student model. By blending insights from models trained on diverse technical domains, Nvidia ensured that Nemotron 3 Ultra could generalize better across different types of coding tasks. The company published its reinforcement learning blends and instructions for reproducing the pipeline on GB200 NVL72 hardware, allowing others to verify the results.
Key details
- Nvidia generated approximately 20,000 rows of post-training data using Factory’s Droid harness.
- Nemotron 3 Ultra’s Terminal-Bench 2.0 score increased from 38.5% to 49% after post-training.
- A preliminary checkpoint of Nemotron 3.5 Super VL improved from 52.11% to 60.4% on Terminal-Bench 2.1.
- PivotRL outperformed standard supervised fine-tuning by an average of 4.17% across four agentic domains.
- The training pipeline included data from OpenCode, Codex, and an environment identified as
agent006. - Factory has not published an ablation study isolating its specific contribution to the benchmark gains.
Why it matters
For engineers building AI agents, this case highlights the diminishing returns of scaling data volume without improving data quality. Instead of feeding models billions of generic tokens, Nvidia demonstrated that a small set of 20,000 carefully curated examples targeting specific failure modes can yield significant performance gains. This suggests that future improvements in agentic AI will depend more on precise error analysis and targeted reinforcement learning than on brute-force pre-training.
The results also underscore the importance of evaluation harnesses in shaping model behavior. Performance gains were closely tied to the specific environments used for training and testing. As seen with Google’s Gemini 4 Neon, which performed better inside its own Antigravity harness, models can become optimized for specific testing frameworks. Developers must be cautious when interpreting benchmark scores, as they may reflect harness-specific adaptation rather than general capability improvements.
What you can do
- Instrument your agent workflows to capture reproducible failures in tool calling and error recovery.
- Prioritize creating small, high-quality datasets that target specific weak points identified in evaluation.
- Consider using outcome-based reward mechanisms like PivotRL instead of strict reference matching for agentic tasks.
- Vary your evaluation environments to prevent your model from overfitting to a single testing harness.
- Explore multi-teacher distillation to transfer specialized capabilities from domain-specific models into your general-purpose agent.
- Review published reinforcement learning blends and pipelines to understand the trade-offs between different post-training methods.



