AWS Bedrock Knowledge Base Routing: How CDK Deploys Seven-Layer Observability for Multi-KB Agent Retrieval

AWS published a CDK implementation pattern for agentic retrieval that routes queries across multiple knowledge bases, instruments seven observability layers, and supports both on-demand and continuous evaluation. The architecture exposes how to wire together knowledge base selection, citation grounding, and production monitoring in a single deployable unit.

This is not a tutorial on setting up a single knowledge base. It is a reference implementation for production-grade multi-KB routing with full observability and evaluation pipelines.

Architecture: AgentCore Gateway and Dual Knowledge Bases

The CDK application deploys an agent that queries two separate Bedrock knowledge bases. The AgentCore Gateway sits between the agent runtime and the knowledge bases, handling routing decisions and response assembly.

Routing flow:

1. Agent receives user query
2. Gateway evaluates query intent and selects target knowledge base (or both)
3. Retrieval executes against selected KB(s)
4. Gateway assembles grounded response with citations
5. Observability layers capture metrics at each stage

The gateway does not use a separate LLM call for routing. It applies rule-based logic or embedding similarity to decide which knowledge base to query. For queries that span domains, it can fan out to both KBs and merge results.

Knowledge base structure:

• KB1: Domain-specific corpus (e.g., financial regulations)
• KB2: General reference corpus (e.g., internal documentation)
• Both backed by OpenSearch Serverless vector stores
• Separate embedding models per KB if domain vocabularies differ

The CDK stack provisions both knowledge bases, the agent runtime, and the gateway as a single deployment. No manual wiring between components.

Seven Observability Layers

AWS breaks observability into seven instrumentation points. Each layer captures a different failure mode or performance bottleneck.

Layer	Metric	Purpose
1. Query ingestion	Request count, latency	Detect traffic spikes or client timeouts
2. Routing decision	KB selection distribution	Identify routing bias or logic errors
3. Retrieval execution	Vector search latency, result count	Catch slow queries or empty result sets
4. Grounding	Citation count, grounding confidence	Measure answer quality and hallucination risk
5. Response assembly	Token count, assembly time	Track cost and latency of final response
6. Agent runtime	Invocation errors, throttles	Monitor Bedrock API limits and failures
7. End-to-end	User satisfaction, task completion	Correlate technical metrics with outcomes

Each layer emits CloudWatch metrics. The CDK stack creates dashboards that visualize all seven layers in a single view.

Key observability patterns:

• Layers 1-3 instrument the retrieval path
• Layers 4-5 instrument the generation path
• Layer 6 monitors the Bedrock service boundary
• Layer 7 ties technical metrics to business outcomes

The stack also provisions X-Ray tracing across all components. Every query generates a trace that shows routing decision, retrieval latency, and grounding time.

Continuous vs. On-Demand Evaluation

The CDK application supports two evaluation modes. Both use the same evaluation dataset but trigger at different times.

On-demand evaluation:

• Triggered manually or via CI/CD pipeline
• Runs full evaluation suite against a snapshot of the agent
• Measures retrieval accuracy, citation quality, and answer correctness
• Outputs evaluation report to S3
• Use case: Pre-deployment validation, A/B testing

Continuous evaluation:

• Runs automatically on a sample of production traffic
• Evaluates a percentage of queries (e.g., 5%) in real time
• Measures same metrics as on-demand but on live data
• Alerts on metric degradation
• Use case: Drift detection, production monitoring

Both modes use the same CDK construct. You configure the evaluation trigger (manual, scheduled, or sampling rate) and the evaluation dataset location.

Evaluation metrics:

• Retrieval precision: Did the agent retrieve relevant documents?
• Citation accuracy: Do citations match retrieved sources?
• Answer correctness: Does the answer align with ground truth?
• Grounding rate: What percentage of answers include citations?

Continuous evaluation writes metrics to CloudWatch. You can set alarms on retrieval precision or grounding rate to catch degradation before users notice.

CDK Constructs and Deployment Shape

The CDK application uses L3 constructs that bundle multiple resources into a single logical unit. You do not provision individual Lambda functions or IAM roles. You instantiate high-level constructs that handle wiring.

Core constructs:

import { BedrockKnowledgeBase, AgentRuntime, ObservabilityStack } from '@aws-cdk/aws-bedrock-agents';

const kb1 = new BedrockKnowledgeBase(this, 'FinancialKB', {
  embeddingModel: 'amazon.titan-embed-text-v1',
  vectorStore: 'opensearch-serverless',
  dataSource: s3Bucket.arnForObjects('financial-docs/*'),
});

const kb2 = new BedrockKnowledgeBase(this, 'GeneralKB', {
  embeddingModel: 'amazon.titan-embed-text-v1',
  vectorStore: 'opensearch-serverless',
  dataSource: s3Bucket.arnForObjects('general-docs/*'),
});

const agent = new AgentRuntime(this, 'MultiKBAgent', {
  knowledgeBases: [kb1, kb2],
  routingStrategy: 'intent-based',
  foundationModel: 'anthropic.claude-3-sonnet',
});

const observability = new ObservabilityStack(this, 'Observability', {
  agent: agent,
  layers: ['ingestion', 'routing', 'retrieval', 'grounding', 'assembly', 'runtime', 'outcome'],
  dashboardName: 'MultiKBAgentDashboard',
});

Deployment steps:

1. cdk synth generates CloudFormation templates
2. cdk deploy provisions all resources in dependency order
3. Knowledge bases ingest data from S3
4. Agent runtime registers with Bedrock
5. Observability stack creates dashboards and alarms

The entire stack deploys in 10-15 minutes. No manual configuration of IAM policies, VPC networking, or service integrations.

Resource dependencies:

• Knowledge bases depend on S3 buckets and OpenSearch domains
• Agent runtime depends on knowledge bases and foundation model access
• Observability stack depends on agent runtime and CloudWatch Logs

CDK handles all dependency ordering. You cannot deploy the agent before the knowledge bases exist.

State Management and Failure Modes

The agent is stateless. Each query is independent. The gateway does not maintain conversation history or user context.

State boundaries:

• Knowledge bases: Stateful (vector index persists in OpenSearch)
• Agent runtime: Stateless (no session memory)
• Gateway: Stateless (no routing cache)
• Observability: Stateful (metrics and traces persist in CloudWatch)

Likely failure modes:

1. Routing logic error: Gateway selects wrong KB or fails to fan out. Observability layer 2 detects skewed KB selection distribution.
2. Empty retrieval: Query returns no results from either KB. Layer 3 emits zero result count. Agent falls back to foundation model without grounding.
3. Citation mismatch: Grounding layer generates citations that do not match retrieved documents. Layer 4 emits low grounding confidence. Continuous evaluation flags citation accuracy drop.
4. Bedrock throttle: Agent hits Bedrock API rate limit. Layer 6 emits throttle errors. Gateway retries with exponential backoff.
5. Evaluation drift: Continuous evaluation detects retrieval precision drop. CloudWatch alarm triggers. Operator investigates knowledge base staleness or query distribution shift.

The CDK stack provisions dead-letter queues for failed invocations and CloudWatch alarms for each failure mode. You configure alarm thresholds in the observability construct.

Security Boundaries

The agent operates within a single AWS account. Knowledge bases, agent runtime, and gateway share an IAM execution role.

Access control:

• Knowledge bases: IAM policies restrict which S3 prefixes the agent can ingest
• Agent runtime: Resource-based policy limits which principals can invoke the agent
• Gateway: No direct external access (invoked only by agent runtime)
• Observability: CloudWatch Logs encrypted at rest, access controlled via IAM

Data flow security:

• Queries and responses transit over TLS
• Vector embeddings stored in OpenSearch with encryption at rest
• Citations include source document metadata but not full document content
• Evaluation datasets stored in S3 with bucket policies

The CDK stack does not provision VPC networking by default. Knowledge bases and agent runtime use public Bedrock endpoints. For private deployments, you add VPC constructs and configure PrivateLink endpoints.

Technical Verdict

Use this pattern when:

• You need to route queries across multiple knowledge bases with different domains
• You require production-grade observability with metric correlation across retrieval, grounding, and generation
• You want infrastructure-as-code deployment with no manual service wiring
• You need both pre-deployment validation and continuous production evaluation

Avoid this pattern when:

• You have a single knowledge base (the routing layer adds unnecessary complexity)
• You need stateful conversation memory (the agent is stateless by design)
• You require sub-second latency (multi-KB routing and grounding add 200-500ms overhead)
• You need fine-grained access control per knowledge base (the agent shares a single execution role)

The seven-layer observability model is the most useful part of this implementation. Even if you do not need multi-KB routing, the instrumentation pattern applies to any agentic retrieval system.

---

Source Links

• AWS Blog: Build agentic retrieval on a Managed Amazon Bedrock Knowledge Base with CDK