Using database state as a reward signal for agent training

Microsoft and Hugging Face have released ThinkingBox-Training, an open-source framework that uses the final state of a database to grade and train AI agents. Published in October 2026, the project demonstrates how executable checks can serve as precise reward signals for reinforcement learning, significantly boosting agent dependability without requiring human-labeled data.

What happened

The teams built ThinkingBox to address a common failure mode in autonomous agents: models that claim a task is complete while leaving the underlying database in an incorrect state. In their initial benchmark, ThinkingBox-Bench, they evaluated eighteen models across 507 tasks involving real backends. They found that high average success rates often masked poor consistency. For instance, Kimi-K3 solved 93.89% of tasks at least once but only achieved perfect consistency on 13.41% of them. This gap highlighted that headline accuracy metrics like pass@1 do not reliably predict production readiness where repeated correctness is essential.

To improve this, the researchers used the same executable grading mechanism from the benchmark as a reward function for post-training. They selected a pool of tasks outside the benchmark dataset and fine-tuned the Qwen3.8-27B model using Group Relative Policy Optimization (GRPO) for 50 steps. The training focused on "offline policy-relative difficulty," meaning they pre-selected tasks that were neither too easy nor impossible for the base model to ensure the learning signal contained enough contrast. The result was a sharp increase in repeatable success, with the number of tasks solved correctly in all twenty attempts rising from 38 to 107.

How it works

ThinkingBox evaluates agents by provisioning a real backend and a set of Model Context Protocol (MCP) tools for each task. Instead of checking if the agent called specific functions, the system inspects the terminal state of the database after the agent finishes. If the records match the expected outcome, the agent receives a reward of 1; otherwise, it receives 0. This method incentivizes the model to find any valid solution path rather than mimicking a specific human-defined sequence of tool calls.

For training, the team employed a technique called offline policy-relative difficulty selection. Before training began, they ran candidate tasks against the base Qwen3.8-27B model to estimate success probabilities. They kept only tasks that produced mixed results—some successes and some failures—because GRPO requires variance within a group of rollouts to learn. If every attempt in a group fails or succeeds, the relative reward signal is zero, providing no gradient for improvement. This fixed pool of 187 tasks was then used for 50 steps of full-parameter RL fine-tuning on 24 H100 GPUs.

Key details

• Model improvement: Qwen3.8-27B saw its pass@1 score rise from 51.70% to 60.78% after 50 GRPO steps.
• Consistency gain: The number of tasks passed in all 20 attempts increased from 38 (7.50%) to 107 (21.10%), nearly tripling dependability.
• Cost efficiency: The estimated cost per dependable task dropped by 67.2%, from $24.36 to $8.00, due to higher success rates reducing wasted retries.
• Benchmark isolation: The 507 tasks in ThinkingBox-Bench were strictly reserved for evaluation and were not included in the training pool.
• Trade-off: While repeatability improved, the pass@20 metric (tasks solved at least once) slightly declined from 89.35% to 87.97%, suggesting the model became more conservative.
• Infrastructure: Training used token-mean GRPO with 18 groups of 8 rollouts per update, handling system errors by refilling groups rather than penalizing the agent.

Why it matters

For engineers building agentic workflows, this work underscores that average accuracy is a misleading metric for production systems. A model that gets the right answer 90% of the time but fails catastrophically on critical edge cases is often less useful than a slightly less accurate model that is consistently reliable. By shifting the focus to "observed 20/20" success rates, developers can better assess whether an agent is ready to handle sensitive operations like financial transactions or inventory updates without constant human supervision.

The approach also offers a practical path to improving open-weight models without expensive human annotation. Using executable code and database states as ground truth allows for scalable, automated reward modeling. This means teams can fine-tune smaller, cost-effective models like Qwen3.8-27B to perform comparably to larger proprietary models on specific, well-defined tasks, potentially reducing inference costs while maintaining high reliability standards.

What you can do

• Evaluate your current agents using state-based checks rather than just output text matching to catch silent failures.
• Consider fine-tuning open-weight models with GRPO if you have a deterministic way to verify task completion, such as database queries or API status checks.
• Filter training tasks by difficulty relative to your base model to ensure the reinforcement learning signal contains sufficient variance for learning.
• Monitor both pass@1 and consistency metrics (like pass@20 or observed 20/20) to get a complete picture of agent reliability.
• Use isolated backends for testing to prevent agents from affecting production data during evaluation and training phases.
• Explore the ThinkingBox-Training repository on GitHub to implement similar executable reward loops for your specific domain tasks.