Four layers of context engineering for production AI agents

Production AI agents often fail during complex tasks like data analysis because they exhaust their context windows. A new reference implementation from Amazon Web Services, called Deep Insight, outlines a four-layer strategy to solve this bottleneck. Published on November 4, 2026, the guide details how to isolate, control, externalize, and verify context to maintain accuracy and reduce costs.

What happened

Large language models offer generous context windows, such as the 200K tokens available in Anthropic Claude Sonnet 4.5 on Amazon Bedrock. However, real-world workflows quickly consume this capacity. A typical request to analyze sales data and generate a report involves multiple stages, including data loading, statistical validation, and chart generation. These steps can burn through over 150K tokens in just 15 minutes, leading to mid-execution failures, degraded accuracy due to stale information, and higher costs.

To address this, AWS engineers developed Deep Insight, an open-source multi-agent solution built on the Strands Agents SDK and Amazon Bedrock AgentCore. The system applies a structured approach to context engineering, moving beyond simple prompt tweaks to a holistic architectural discipline. The implementation shows that combining these techniques can cut the tokens added by each script by about 95 percent and shrink system prompts by roughly 97 percent.

The guidance is organized into four distinct layers: isolating context at the architecture level, controlling it through prompts, externalizing it to the filesystem, and verifying it with independent safeguards. This layered method ensures that when one technique reaches its limit, the next layer compensates, preventing the "context rot" that plagues single-agent systems.

How it works

The first layer focuses on architectural isolation. Instead of running all tasks in a single agent, Deep Insight uses a hierarchy of specialized agents, including a Coordinator, Planner, Supervisor, and various Tool Agents. Each agent operates in its own independent context window. They communicate via a shared state dictionary that contains only compressed summaries, or "clues," rather than full conversation histories. This ensures that even the most active agents, such as the Coder, stay under 25K tokens.

The second layer controls context inflow through strict prompt engineering. Each agent has explicit output token budgets. For example, the Coder agent is limited to 1,500 tokens because its actual code is saved to files, not returned in the chat. Prompts also enforce a self-contained code philosophy, requiring every script to include its own imports and data loading logic. This prevents the need to keep previous code snippets in memory, allowing old context to be discarded safely.

The third and most impactful layer externalizes heavy content to the filesystem. Deep Insight replaces separate write and execute tools with a single write_and_execute_tool. This tool writes Python code to a file and runs it immediately, returning only a brief success summary to the agent. The code itself never enters the context window. Additionally, common logic is stored in reusable modules, and detailed analysis results are saved to structured text files like all_results.txt. Agents read these files only when necessary, keeping the active context lean.

Key details

• Deep Insight reduces script context usage by approximately 95 percent using a combined write-and-execute tool.
• Loading skills on demand shrinks the system prompt size by about 97 percent.
• The system uses a shared_state dictionary to pass only compressed clues and completion notices between agents.
• Coder agents are restricted to 1,000–1,500 output tokens, forcing them to summarize rather than dump code.
• Common utility functions are pre-generated into a coder_analysis_utils.py module to avoid regenerating duplicate code.
• Structured note-taking files like all_results.txt serve as an inter-agent communication bus, replacing verbose message passing.

Why it matters

For engineers building production AI systems, context management is no longer just a performance optimization; it is a reliability requirement. Without these layers, agents suffer from attention degradation as the context window fills. Important instructions get lost, and the model begins to hallucinate or ignore recent constraints. By isolating contexts, teams can ensure that each sub-task receives the full attention of the model without interference from unrelated history.

Cost efficiency is another critical driver. In per-token pricing models, sending hundreds of thousands of tokens for routine tasks is unsustainable. The file-based externalization strategy demonstrated in Deep Insight shifts the burden from expensive API calls to cheap storage operations. This allows developers to build more complex, long-horizon agents without prohibitive cost increases.

Finally, this approach improves debuggability and maintenance. When code and results are stored in files rather than hidden in ephemeral chat history, developers can inspect artifacts directly. The separation of concerns means that errors in one agent do not necessarily corrupt the state of another, making the system more resilient to partial failures.

What you can do

• Adopt a multi-agent architecture where specialized workers handle distinct tasks with isolated context windows.
• Implement strict output token budgets in your system prompts to prevent agents from generating verbose, unnecessary text.
• Replace separate code-writing and execution tools with a single tool that writes to disk and returns only execution summaries.
• Enforce self-contained code generation by requiring agents to include all imports and data loading in every script.
• Use the filesystem as a communication bus by saving intermediate results to structured files that downstream agents can read.
• Pre-generate common utility modules to avoid repeating boilerplate code across multiple agent turns.