Voice agents are moving from demos to production. The shift exposes a problem that text-based agents can ignore: how do you prevent one caller’s conversation context from bleeding into another’s when sessions overlap in real time?
AWS just published architectural patterns for production voice agents using Amazon Nova Sonic, Bedrock AgentCore Runtime, and Strands BidiAgent. The patterns address session segmentation, state isolation, and tool permission boundaries. This is the plumbing that makes voice agents safe in multi-tenant environments.
The Session Isolation Problem
Voice agents handle bidirectional audio streams. Unlike request-response APIs, a single conversation can span minutes and involve multiple tool invocations. Three risks emerge:
- Context leakage: User A’s conversation state persists in memory when User B connects.
- Tool permission bleed: A tool authorized for one session remains callable in another.
- Noisy neighbor latency: One session’s heavy processing delays another’s audio response.
Text agents can rely on stateless HTTP handlers. Voice agents need session-scoped isolation at the runtime level.
Bedrock AgentCore Runtime: MicroVM Session Isolation
AgentCore Runtime hosts voice agents as containers. Each session gets a microVM. This is not a shared container pool with logical session IDs. Each incoming WebSocket connection triggers a new microVM instance.
What this prevents:
- Memory leakage between sessions (each VM has its own heap).
- CPU contention from concurrent tool calls (each VM gets dedicated compute).
- Shared state bugs (no global variables across sessions).
What it costs:
- Cold start latency for the first audio frame (mitigated by pre-warming pools).
- Higher infrastructure overhead than shared-process models.
The trade-off makes sense for voice. A 200ms cold start is acceptable if it prevents a 2-second delay caused by another session’s database query.
Three State Management Patterns
AWS describes three architectural patterns for voice agents. The difference is where conversation state lives.
| Pattern | State Location | Latency | Use Case |
|---|---|---|---|
| Stateless | Reconstructed per turn | Low (no DB reads) | Simple Q&A, no multi-turn context |
| Session-scoped | In-memory during call | Medium (warm cache) | Customer service, order tracking |
| Persistent | DynamoDB/S3 across calls | High (cross-session reads) | Long-running workflows, follow-up calls |
Stateless Pattern
Each turn is independent. The agent receives audio, invokes tools, returns audio. No conversation history.
Plumbing:
- No session store.
- Tool calls are scoped to the current turn only.
- AgentCore destroys the microVM after the response.
Failure mode: User says “What did I just ask?” Agent has no memory.
Session-Scoped Pattern
Conversation state lives in the microVM’s memory. The WebSocket stays open. Tool results accumulate in a session object.
Plumbing:
# Strands BidiAgent manages session state in-memory
class VoiceSession:
def __init__(self, session_id):
self.session_id = session_id
self.context = {}
self.tool_results = []
def add_tool_result(self, tool_name, result):
self.tool_results.append({
"tool": tool_name,
"result": result,
"timestamp": time.time()
})
def get_context(self):
# Context includes all prior tool results
return {
"session_id": self.session_id,
"history": self.tool_results
}Failure mode: WebSocket disconnects. Session state is lost. User must start over.
Persistent Pattern
Conversation state is written to DynamoDB or S3 after each turn. The agent can resume across calls.
Plumbing:
- After each tool invocation, write session state to DynamoDB with a TTL.
- On reconnect, load state by session ID.
- Tool permissions are revalidated on each turn.
Failure mode: High latency on every turn (50-100ms per DynamoDB read). Stale state if writes fail silently.
Tool Permission Boundaries
Nova Sonic integrates with AgentCore Gateway, which hosts tools using the Model Context Protocol (MCP). Tools are registered per session.
How tool isolation works:
- Session starts. AgentCore creates a microVM.
- Agent declares required tools (e.g.,
get_order_status,cancel_order). - AgentCore Gateway provisions tool endpoints scoped to the session ID.
- Tool calls include a session token. Gateway validates the token before execution.
- Session ends. Gateway revokes tool access.
Security boundary: A tool invoked in Session A cannot be called by Session B, even if Session B knows the tool name. The session token is tied to the microVM instance.
Observable risk: If the session token is logged or leaked, an attacker can replay tool calls. Tokens should be short-lived (5-10 minutes) and rotated on reconnect.
Strands BidiAgent: Bidirectional Stream Management
Strands BidiAgent is an open-source framework that handles the WebSocket lifecycle between Nova Sonic and your application logic.
What it does:
- Manages the SigV4-authenticated WebSocket to Nova Sonic.
- Routes incoming audio to the model.
- Routes tool call requests to your handler functions.
- Streams audio responses back to the user.
What it does not do:
- Enforce tool permissions (that’s AgentCore Gateway’s job).
- Persist session state (you choose stateless, session-scoped, or persistent).
- Handle reconnects (you must implement retry logic).
Code shape:
from strands import BidiAgent
agent = BidiAgent(
model="nova-sonic",
tools=[get_order_status, cancel_order],
session_store=DynamoDBSessionStore() # Optional
)
# Start WebSocket connection
agent.start_session(session_id="user-123")
# BidiAgent routes tool calls to your functions
@agent.tool
def get_order_status(order_id: str) -> dict:
# This runs in the session's microVM
return {"status": "shipped", "eta": "2026-05-22"}Observability: Voice-Specific Metrics
AgentCore Runtime exposes telemetry for voice agents. Key metrics:
- Time-to-first-audio: Latency from user speech to agent response start.
- Tool invocation latency: Time spent in tool calls per session.
- Session duration: Total WebSocket connection time.
- MicroVM cold start time: Delay before the first audio frame.
What to alert on:
- Time-to-first-audio > 500ms (user perceives lag).
- Tool invocation latency > 2 seconds (conversation feels broken).
- MicroVM cold start > 300ms (pre-warming pool is undersized).
Deployment Shape
A production voice agent deployment on AgentCore looks like this:
- Container image: Package your agent code, Strands BidiAgent, and tool handlers.
- AgentCore Runtime: Deploy the container. AgentCore provisions a microVM pool.
- AgentCore Gateway: Register tools via MCP. Gateway handles session-scoped access.
- DynamoDB (optional): Store persistent session state with a TTL.
- CloudWatch: Collect voice-specific metrics.
Scaling behavior:
- AgentCore auto-scales microVMs based on WebSocket connection count.
- Each microVM handles one session.
- If you have 100 concurrent calls, you have 100 microVMs.
Cost model: You pay per microVM-second, not per request. A 5-minute call costs more than 10 one-second calls.
Likely Failure Modes
Session token replay: If an attacker captures a session token, they can invoke tools until the token expires. Mitigation: Use short TTLs and rotate tokens on reconnect.
State desync in persistent mode: If a DynamoDB write fails silently, the agent loses context. Mitigation: Use conditional writes and retry logic.
Noisy neighbor in shared-process models: If you skip microVM isolation and use a shared container, one session’s heavy tool call can delay another’s audio response. Mitigation: Use AgentCore’s microVM isolation.
WebSocket disconnect without cleanup: If the client disconnects abruptly, the microVM may linger until a timeout. Mitigation: Set aggressive WebSocket timeouts (30-60 seconds).
Technical Verdict
Use this architecture when:
- You need multi-turn voice conversations with tool invocations.
- You have multiple concurrent users and cannot risk context leakage.
- You can tolerate 200-300ms cold start latency for session isolation.
- You need observable, session-scoped tool permissions.
Avoid this architecture when:
- Your voice agent is stateless (single-turn Q&A). Use a simpler request-response model.
- You need sub-100ms latency for every turn. MicroVM overhead will hurt.
- You are building a single-user prototype. The isolation overhead is unnecessary.
Session segmentation is not optional for production voice agents. The question is whether you implement it at the runtime level (microVMs) or the application level (session IDs in a shared process). AgentCore chooses the runtime level. The trade-off is higher infrastructure cost for guaranteed isolation.