Building with AI

Optimizing Amazon Bedrock AgentCore V2 cold starts with snapshotting

Amazon Bedrock AgentCore V2 uses microVM snapshots to reduce session startup latency. Engineers can optimize performance by shifting initialization work to the build phase.

A glass cube with a frozen circuit board inside, representing a microVM snapshot releasing data streams.
Illustration generated for this article

Amazon Web Services has updated the Amazon Bedrock AgentCore runtime to version V2, introducing a snapshot-based architecture designed to accelerate session startup times for AI agents. Released in November 2026, this update shifts the initialization burden from every new user session to a one-time build process, significantly reducing latency for interactive applications. The change requires developers to restructure how their agents handle imports, client connections, and static data loading to fully benefit from the new memory preservation capabilities.

What happened

The Amazon Bedrock AgentCore runtime provides a serverless environment where each agent session runs in an isolated microVM. In the previous V1 architecture, every new session required pulling a container image from Amazon Elastic Container Registry, starting the container, and running initialization code before processing the first request. This process meant that cold start times grew directly with image size, creating noticeable delays for users waiting for the first response.

V2 changes this flow by performing the heavy lifting only once. When a runtime is created or updated, the platform starts the container, waits for a healthy ping, and then freezes the microVM state into a snapshot. This snapshot includes the memory and disk state of the initialized application. For subsequent sessions, the platform simply restores this snapshot into a new microVM, allowing the process to resume exactly where it left off. This approach reclaims unused memory and ensures that session start times are faster and more consistent, though it does increase the time required for runtime creation or updates.

How it works

The core mechanism relies on capturing the state of the application after initialization but before it begins handling user traffic. During the build phase, the runtime clears the operating system’s file cache to keep the snapshot small, then freezes the microVM. When a new session arrives, the snapshot is restored, bringing all imports, objects, and clients back into memory instantly. However, file reads do not carry over because the cache was cleared; instead, files from the container image are fetched lazily when first accessed.

This behavior creates distinct performance characteristics for different types of work. Memory-resident data, such as loaded libraries and instantiated clients, is available at memory speed immediately. Network connections, however, become stale after restoration and must be reconnected. Developers must distinguish between static work, which can be snapshotted, and dynamic data, which cannot. To assist with complex setups, V2 introduces build hooks that allow agents to prime external tools or load large files into memory during the snapshot creation process, ensuring they are ready for immediate use upon session restore.

Key details

  • Snapshot timing: The snapshot is built once per create or update operation, taking a few minutes, rather than on every session start.
  • Memory preservation: Objects and imports held in memory at snapshot time are restored instantly, while file caches are cleared to minimize snapshot size.
  • Lazy file loading: Files from the container image are fetched on first read, meaning large files not held in memory will incur latency on first access.
  • Build hooks: New HTTP endpoints allow agents to perform additional warmup tasks, such as loading browser binaries or CLI tools, during the snapshot build.
  • Metrics: Platform overhead is tracked via OverheadLatency in CloudWatch, while total time to first byte is tracked via Latency.
  • Network handling: Stale network sockets must be closed before snapshotting to avoid connection errors, though some clients like boto3 reconnect automatically.

Why it matters

For software engineers building interactive AI experiences, such as chatbots or voice assistants, even two seconds of delay can degrade user satisfaction. The V2 architecture offers a path to sub-second startup times, but only if the application is structured correctly. Moving initialization work from the request handler to the build phase reduces the per-session cost, but it increases the memory footprint of each session. Developers must balance these trade-offs, ensuring that the data held in memory justifies the increased resource usage.

Understanding the distinction between init time and request latency is critical for effective optimization. Init time now contributes to the deployment duration rather than the user-facing latency. By measuring OverheadLatency and separating it from application logic time, teams can identify whether delays stem from the platform restoring the snapshot or from inefficient code executing within the handler. This visibility allows for targeted improvements, such as pre-loading large models or compiling bytecode during the build phase.

What you can do

  • Shift initialization to startup: Move imports, client creation, and static data loading to occur before app.run() so they are captured in the snapshot.
  • Use build hooks for external tools: Implement HTTP endpoints to load large binaries, such as browsers or CLIs, into memory during the snapshot build process.
  • Keep static data in memory: Read large reference files into objects during init rather than reading them from disk on every request.
  • Manage network connections: Close long-lived network connections before the snapshot is taken to prevent stale socket errors upon restoration.
  • Compile bytecode explicitly: Set environment variables like UV_COMPILE_BYTECODE=1 to ensure dependencies are compiled during init, avoiding compilation delays in new sessions.
  • Monitor specific metrics: Track OverheadLatency to measure platform restore time and subtract it from total Latency to isolate application performance.
  • #aws
  • #serverless
  • #ai-agents
  • #performance
  • #bedrock

Tools from the Bytechap store

$89

DocBento

Self-hosted document management that reads every scan and answers with page citations.

Live demo

Keep reading

All stories