Production AI agents fail when they lose track of state. The problem is not the model. It is context management: how you pass information between agents, store intermediate results, and prevent token budgets from exploding as tasks grow.
AWS published working code from Deep Insight, an open-source multi-agent data analysis solution built on Strands SDK and Bedrock AgentCore. The implementation exposes a four-layer context engineering framework that divides responsibility across architecture, prompts, file-based tools, and verification agents. This is not theory. It is production plumbing with clear handoff boundaries.
The Four Layers
Context engineering is a layered architecture problem. Each layer handles a different scope of state:
Layer 1: Architecture
The orchestration shape determines which agents see which context. Deep Insight uses a hub-and-spoke model with a coordinator agent that routes tasks to specialist agents (data loader, analyst, visualizer). The coordinator holds the task graph. Specialists hold only their slice of work. This prevents every agent from needing the full conversation history.
Layer 2: Prompts
System prompts embed static context: role definitions, output format constraints, and tool usage rules. Deep Insight prompts include explicit instructions on when to call tools versus when to return results. This keeps the agent from hallucinating intermediate steps or skipping verification.
Layer 3: File-Based Tools
Agents write intermediate results to files instead of passing them as message content. A data loader agent writes a CSV to S3. An analyst agent reads that file path, not the raw data. This decouples token cost from data size. A 10MB dataset costs the same as a 10-byte file path in the context window.
Layer 4: Verification Agents
A separate agent validates outputs before they move to the next stage. Deep Insight runs a verification agent after the analyst produces a chart. The verifier checks schema compliance, null handling, and axis labels. If verification fails, the coordinator re-routes the task. This prevents cascading errors from bad intermediate state.
Handoff Mechanics
Context transitions happen at tool boundaries. When the coordinator assigns a task, it passes:
- Task ID
- Input file path (not file content)
- Expected output schema
- Verification criteria
The specialist agent:
- Reads the input file from storage
- Executes the task
- Writes output to a new file
- Returns the output file path and metadata
The coordinator then routes the output path to the next agent or to the verification layer. No agent holds more than its current task in memory.
File-Based Tool Storage vs. In-Memory Context
| Approach | Token Cost | Persistence | Restart Behavior | Observability |
|---|---|---|---|---|
| In-memory context | Grows with data size | Lost on crash | Restart from scratch | Logs only |
| File-based tools | Fixed (file path) | Survives restarts | Resume from last checkpoint | Files + logs |
File-based tools cost more in I/O latency but buy you token efficiency and crash recovery. If an agent dies mid-task, the coordinator can restart it with the same input file path. The intermediate state is already on disk.
Implementation: Coordinator Agent with File Handoff
# Coordinator agent routes tasks and manages file-based context
class CoordinatorAgent:
def __init__(self, s3_bucket, bedrock_client):
self.s3_bucket = s3_bucket
self.bedrock = bedrock_client
self.task_graph = {}
def route_task(self, task_id, task_type, input_path):
# Assign task to specialist agent
specialist = self.get_specialist(task_type)
# Pass only file path, not content
response = specialist.invoke({
"task_id": task_id,
"input_path": input_path,
"output_bucket": self.s3_bucket
})
# Specialist writes output to S3, returns path
output_path = response["output_path"]
# Route to verification agent
verified = self.verify_output(output_path, task_type)
if not verified:
# Re-route task with error context
return self.route_task(task_id, task_type, input_path)
# Update task graph with output location
self.task_graph[task_id] = {
"status": "complete",
"output": output_path
}
return output_path
def verify_output(self, output_path, task_type):
verifier = VerificationAgent(self.bedrock)
result = verifier.check(output_path, task_type)
return result["valid"]The coordinator never loads file content into its context. It passes paths. The specialist agents read and write files directly. This keeps the coordinator’s token budget constant regardless of data size.
State Management Across Agent Restarts
File-based tools enable stateful recovery. The task graph lives in a persistent store (DynamoDB in Deep Insight’s case). If the coordinator crashes:
- New coordinator instance loads task graph from DynamoDB
- Identifies incomplete tasks
- Re-routes tasks using existing input file paths
- Specialist agents pick up where they left off
No need to replay the full conversation history. The file system is the source of truth for intermediate state.
Observability and Debugging
Each layer exposes different observability signals:
- Architecture layer: Task graph state in DynamoDB, agent assignment logs
- Prompt layer: Model invocation logs with input/output tokens
- File-based tools: S3 object metadata, file size, creation timestamps
- Verification agents: Pass/fail decisions, error reasons, retry counts
Deep Insight logs every file write and verification decision. When a task fails, you can inspect the input file, the output file, and the verification error without reconstructing the conversation.
Failure Modes
File path leaks
If an agent hallucinates a file path instead of calling the file-write tool, downstream agents fail. Mitigation: strict output schema validation in prompts and verification agents.
Stale file references
If the coordinator retries a task but points to an old input file, the specialist agent processes outdated data. Mitigation: include task version in file paths (task-123-v2.csv).
Verification bottleneck
If the verification agent is slow, the coordinator queues up tasks. Mitigation: run verification agents in parallel or use async invocation with callbacks.
Token budget on coordinator
The coordinator’s task graph grows with the number of tasks. If you run 1,000 tasks, the task graph metadata can exceed the context window. Mitigation: paginate task graph queries or use a summary agent to compress completed task metadata.
When to Use This Pattern
This four-layer approach works when:
- You have multi-step workflows with large intermediate data
- You need crash recovery without replaying full conversation history
- You want to decouple token cost from data size
- You can tolerate file I/O latency (typically 50-200ms per read/write)
Avoid this pattern when:
- Your tasks are stateless and complete in one LLM call
- Intermediate data is small (under 1KB) and fits in message content
- You need sub-50ms latency and cannot afford file I/O
- Your agents run in environments without persistent storage
Technical Verdict
Deep Insight’s four-layer context framework is production-grade plumbing for multi-agent systems with large intermediate state. The architecture layer controls orchestration shape. The prompt layer embeds static rules. The file-based tool layer decouples token cost from data size. The verification layer prevents cascading errors.
Use this when you are building multi-step agentic workflows that process data larger than a few kilobytes. The file I/O overhead is worth it for token efficiency and crash recovery. Skip it if your agents are stateless or your data fits comfortably in message content.
The code is open source. You can run it on Bedrock AgentCore or adapt it to other orchestration frameworks. The key insight is not the specific tools. It is the layered responsibility model: architecture for routing, prompts for rules, files for data, verification for quality gates.
Source Links
- AWS ML Blog: Four Layers of Context Engineering
- Deep Insight on GitHub (implementation reference)