Training RL adapters inside agent harnesses boosts accuracy and cuts token costs
Hugging Face shows how training LoRA adapters with reinforcement learning directly inside agent harnesses improves Qwen3.8-27B performance on coding tasks while reducing token usage by half.
Hugging Face released a technical guide demonstrating how to train custom reinforcement learning adapters for open models within specific agent harnesses. Published in November 2026, the work focuses on optimizing the Qwen3.8-27B model for software engineering tasks using three popular harnesses: Pi, OpenCode, and Hermes Agent.
What happened
General-purpose large language models are not fully optimized for every specific agent framework or task distribution. When developers pair a base model with a particular harness, they often leave performance and efficiency gains on the table. Hugging Face addressed this by combining reinforcement learning on a custom task distribution with specific agent harnesses to create tailored adapters that squeeze more capability out of the model for those exact combinations.
The team trained one Low-Rank Adaptation (LoRA) adapter for Qwen3.8-27B per harness. They used reinforcement learning inside the harness itself, ensuring the model learned from the exact tokens produced during real agent interactions. The results showed significant improvements. For example, Qwen3.8-27B’s score on Terminal-Bench 2.1 with the Pi harness jumped from 39% to 55% pass@1. More importantly, average token usage per task dropped from 2.9K to 1.5K. This efficiency gain came from the model reasoning less, calling fewer tools, and performing better on the first attempt.
To make this reproducible, Hugging Face shared a fully reusable recipe. All datasets, training configurations, and agent rollouts were executed on the Hub using Jobs, TRL’s AsyncGRPOTrainer, OpenEnv, and Harbor. The final artifacts, including the adapters and GGUF files, are available in a public collection, allowing engineers to apply the same method to their own tasks and harnesses.
How it works
The core innovation is training the model inside the agent harness rather than using a re-implementation of the agent loop. Most reinforcement learning frameworks require agents to be written against the framework's own interface, which can introduce discrepancies between training and deployment. In this approach, the harness runs exactly as it ships. A proxy sits between the harness and the vLLM server, recording the exact tokens produced by the model. The trainer then learns from these real-world interactions, ensuring the adapter optimizes for the specific behaviors and output formats of that harness.
Data quality was ensured through a rigorous pipeline. The team combined tasks from SWE-smith, SWE-Gym, and Tasksmith, filtering them using openenv validate. This tool checked for semantic correctness, ensuring that reference solutions earned maximum rewards and that doing nothing scored lower. They also performed a difficulty check, keeping only tasks that the base model solved sometimes but not always. This "qualified" set provides the necessary learning signal for Group Relative Policy Optimization (GRPO), as tasks that are always easy or always hard offer no gradient for improvement.
Training utilized TRL’s AsyncGRPOTrainer, which does not wait for all rollouts to complete before moving to the next optimization step. This asynchronous approach maximized hardware utilization on NVIDIA B200 GPUs. The final reward function combined verifier correctness with a small tool-efficiency bonus, encouraging the model to solve problems with fewer steps. Safety was maintained using the safeagents plugin, which blocked misbehavior during training, with blocked calls decreasing consistently as the model improved.
Key details
- Model and Harnesses: The study focused on Qwen3.8-27B adapted for Pi 1.5.2, OpenCode 2.2.7, and Hermes Agent v0.22.6.
- Performance Gains: The Pi harness adapter improved Terminal-Bench 2.1 scores from 39% to 55% pass@1.
- Efficiency Improvements: Average token usage per task decreased from 2.9K to 1.5K, driven by reduced reasoning steps and tool calls.
- Training Method: Reinforcement learning was conducted inside the harness using TRL’s
AsyncGRPOTrainerand OpenEnv, with a proxy capturing real token outputs. - Data Pipeline: 871 tasks were curated from SWE-smith, SWE-Gym, and Tasksmith, filtered for validity and qualified difficulty to ensure effective GRPO training.
- Super-Adapter: Three separate LoRA adapters were merged into a single super-adapter with rank 96, maintaining high accuracy across all three harnesses due to low cosine similarity between weight updates.
Why it matters
For engineers building AI agents, this approach offers a path to significantly lower operational costs and higher reliability. By reducing token usage by nearly half, organizations can deploy more capable models like Qwen3.8-27B without prohibitive inference bills. The improvement in first-attempt success rates also reduces latency, leading to smoother user experiences in coding assistants and autonomous agents. This is particularly relevant for teams using open models who need to compete with the efficiency of proprietary, closed-source alternatives.
The methodology also demystifies the process of aligning open models with specific tooling environments. By providing a reusable recipe and open-sourcing the artifacts, Hugging Face lowers the barrier to entry for custom model optimization. Developers no longer need to rely solely on general-purpose fine-tuning; they can now tailor models to their specific stack, whether it involves data analysis, code building, or complex software engineering workflows. This shift towards harness-specific optimization could become a standard practice for production-grade AI agents.
What you can do
- Explore the Hugging Face collection containing the trained adapters, GGUFs, and RL tasks to understand the artifact structure.
- Use the provided Docker images with pre-installed harnesses (Pi, OpenCode, Hermes) to replicate the training environment locally or on the Hub.
- Apply the
openenv validatetool to your own custom tasks to filter for semantic correctness and appropriate difficulty before training. - Implement TRL’s
AsyncGRPOTrainerin your workflow to enable asynchronous rollout processing, improving training efficiency on multi-GPU setups. - Consider merging multiple harness-specific LoRA adapters if you support multiple agent frameworks, leveraging the low interference between orthogonal weight updates.
- Integrate the safeagents plugin with your custom regex and pattern rules to prevent model misbehavior during the reinforcement learning phase.



