Building agentic retrieval on AWS Bedrock with CDK and managed knowledge bases

Amazon Web Services has published a technical guide detailing how to build enterprise-grade agentic retrieval systems using Managed Amazon Bedrock Knowledge Bases and the AWS Cloud Development Kit (CDK). Released in late October 2026, this approach moves beyond traditional single-shot retrieval augmented generation (RAG) by integrating reasoning agents that dynamically plan, retrieve, and cite sources across multiple data domains. The solution emphasizes built-in observability and evaluation, addressing the operational complexity inherent in multi-turn AI interactions.

What happened

Traditional RAG systems typically follow a linear path: embed a user question, fetch the nearest data chunks from a single index, and generate an answer. While effective for simple queries, this method often fails when answers require information from multiple corpora or when the relevant source depends on nuanced context. In such cases, a single retrieval step may return incomplete or misleading data, leading to fluent but incorrect responses. To address these limitations, Amazon Bedrock has evolved its Knowledge Base service to support agentic retrieval, where a reasoning agent decides whether to retrieve data, which knowledge base to consult, and whether additional lookups are needed before composing a final answer.

The new architecture relies on the Managed Knowledge Base, a service feature that handles ingestion, storage, indexing, and retrieval automatically. Unlike customer-managed options that require provisioning and scaling vector databases like Amazon OpenSearch Serverless, the managed version eliminates infrastructure overhead. It supports the AgenticRetrieveStream API, which enables multi-turn planning and tool-driven retrieval. This capability is exclusively available on Managed Knowledge Bases, making them the foundational component for this advanced retrieval strategy. The system also integrates Amazon Bedrock AgentCore, which exposes each knowledge base as a Model Context Protocol (MCP) tool, allowing agents to route queries semantically rather than searching a blended index.

A significant shift in this deployment model is the integration of observability and evaluation directly into the infrastructure code. Previous iterations required developers to manually sequence stack deployments and bolt on monitoring tools after the fact. The new CDK application provisions two Amazon CloudWatch dashboards spanning seven layers of telemetry, along with both on-demand and continuous evaluation metrics. This ensures that every step of the agent’s reasoning loop—from token usage to retrieval quality—is instrumented from the start, providing visibility into system behavior and answer groundedness.

How it works

The solution operates through a coordinated workflow involving four main components: the agent runtime, the agent gateway, managed knowledge bases, and observability dashboards. When a user submits a question, the agent hosted on the Amazon Bedrock AgentCore Runtime initiates a reasoning process. It determines which knowledge base is relevant, such as financial or weather data, and calls the corresponding tool via the AgentCore Gateway. The Gateway uses IAM authorization and SigV4 signing to invoke the AgenticRetrieveStream API on the selected Managed Knowledge Base.

Inside the Managed Knowledge Base, the system decomposes the query into sub-queries, retrieves relevant passages from its internally managed data store, and synthesizes a cited answer. If the initial context is insufficient, the agent loops back to retrieve more information before finalizing its response. Throughout this process, the Runtime emits OpenTelemetry spans, token counts, and metrics to Amazon CloudWatch and AWS X-Ray. This data populates seven observability layers, including KB-native metrics, ingestion status, retrieval quality, and evaluation scores, allowing engineers to monitor latency, call volume, and faithfulness of the generated answers.

Key details

• Managed Infrastructure: The solution uses Type: MANAGED Knowledge Bases, removing the need to provision, scale, or patch vector databases like OpenSearch Serverless.
• Agentic Retrieval API: The AgenticRetrieveStream API performs multi-turn planning and internal reranking, returning synthesized, cited answers only available on managed bases.
• CDK Automation: The entire system is deployed as a single Python CDK application with four stacks, where dependencies are derived automatically from construct references.
• Seven Observability Layers: Telemetry covers KB metrics, ingestion, retrieval quality, Gateway metrics, OTEL span trees, token usage, and evaluation scores.
• Evaluation Metrics: The system includes both on-demand and continuous evaluation, measuring faithfulness and correctness to detect answer drift.
• Sample Data: The deployment includes synthetic financial data and a real U.S. Congressional Research Service report on tornadoes for testing.

Why it matters

For software engineers and AI architects, this shift represents a move from building raw retrieval pipelines to orchestrating intelligent agents. By abstracting away the vector database management, teams can focus on the logic of retrieval and the quality of answers rather than infrastructure maintenance. The ability to route queries across distinct knowledge bases ensures that answers are grounded in the correct domain, reducing the risk of hallucinations caused by irrelevant context. This is particularly critical in enterprise settings where data silos are common and accuracy is paramount.

Furthermore, the emphasis on observability addresses a major pain point in production AI systems: the black box nature of agent behavior. Without detailed telemetry, it is difficult to diagnose why an agent failed or to verify that its answers are supported by source material. By provisioning dashboards and evaluation metrics alongside the application, developers gain immediate insight into system performance and reliability. This built-in transparency supports responsible AI use by ensuring that responses are auditable and traceable to their origins.

What you can do

• Deploy the provided CDK application to provision a complete agentic retrieval system with managed knowledge bases and observability dashboards.
• Configure the config.py file to set project names, regions, and embedding models, allowing for multiple independent deployments in one account.
• Use the included driver notebook to drive traffic and populate the seven observability layers with real-time metrics and evaluation scores.
• Test the system with the bundled financial and weather datasets to understand how the agent routes queries and cites sources across different domains.
• Monitor the CloudWatch dashboards to track latency, token usage, and retrieval quality, adjusting agent parameters as needed for production workloads.
• Evaluate the faithfulness and correctness scores to ensure answers remain grounded in source materials and detect any drift in model performance.