Most agent frameworks hide their orchestration logic behind convenience methods. You pass a prompt, get a response, and everything in between (tool selection, retry logic, context assembly, error recovery) happens in code you never see. That works until you need to debug why your agent called the wrong tool, burned through your token budget, or failed silently in production.
The gap between framework demos and production reliability comes down to visibility. When you can’t see the execution loop, you can’t control failure modes, audit tool calls, or tune performance. You’re debugging a black box with logs that weren’t designed for your use case.
What the execution loop actually does
Between receiving a prompt and returning a result, an agent framework runs a loop that looks roughly like this:
- Assemble context: Merge system prompt, user input, conversation history, and available tool definitions into a single LLM request
- Call the model: Send the assembled context and wait for a response
- Parse the response: Extract tool calls, reasoning steps, or final answers from structured output
- Dispatch tools: If the model requested a tool, execute it and capture the result
- Inject results: Add tool outputs back into context and loop again
- Halt or continue: Decide whether to stop (final answer, token limit, max iterations) or keep going
Most frameworks bundle this loop into a single method. You call agent.run() or chain.invoke() and the loop runs until it hits a stopping condition. The problem is that stopping conditions, retry logic, and error handling are baked into the framework’s assumptions, not yours.
Hidden decisions that break in production
Here are the control points frameworks typically hide:
Tool dispatch order: If the model requests three tools in one turn, does the framework run them sequentially, in parallel, or let you choose? Sequential execution is safer but slower. Parallel execution is faster but harder to debug when one tool fails.
Retry boundaries: When a tool call fails (network timeout, API error, malformed output), does the framework retry the tool, retry the entire LLM call, or surface the error to you? Some frameworks silently retry with exponential backoff. Others fail fast and expect you to handle it.
State persistence: Where does conversation history live between turns? In-memory arrays work for demos but break when you need to resume a conversation after a crash or scale across multiple workers. Frameworks that auto-persist to disk or external stores add latency you may not want.
Token budget enforcement: Does the framework count tokens and halt before you hit the model’s context limit, or does it let the LLM reject the request? Pre-flight token counting adds overhead but prevents wasted API calls.
Error propagation: When a tool throws an exception, does the framework pass the error message back to the model (so it can reason about the failure) or halt execution immediately? Passing errors to the model can help it recover, but it also leaks implementation details into the prompt.
Comparing framework control surfaces
| Framework | Execution loop visibility | Tool dispatch control | State location | Retry config | Error handling |
|---|---|---|---|---|---|
| LangChain | Opaque (inside invoke()) | Sequential by default | In-memory or pluggable | Per-chain config | Exception or return |
| AutoGen | Visible (agent conversation loop) | Parallel via async | In-memory | Manual in agent code | Agent decides |
| Semantic Kernel | Opaque (inside planner) | Planner decides | Pluggable memory connectors | Per-function config | Exception or continue |
| Reactive Agents | Explicit (builder pattern) | Developer specifies | Opt-in via .withMemory() | Per-tool override | Developer handles |
| Custom harness | Fully visible | You write it | You choose | You write it | You write it |
The trade-off is always convenience versus control. Opaque loops get you running faster. Visible loops let you tune for your failure modes.
What explicit orchestration looks like
Reactive Agents (the framework referenced in the source article) takes the position that every capability should be opt-in. Here’s what that looks like in practice:
import { ReactiveAgents } from "reactive-agents";
const agent = await ReactiveAgents.create()
.withProvider("anthropic")
.withModel("claude-sonnet-4-6")
.withReasoning() // Enables chain-of-thought prompting
.withTools({
tools: [getServiceHealth, getRecentDeploys],
parallel: false, // Sequential execution
retryOnError: true,
maxRetries: 2
})
.withMemory({
provider: "redis",
ttl: 3600
})
.withGuardrails({
input: [validateNoSecrets],
output: [checkForPII]
})
.build();
const result = await agent.run(
"The payments-api is alerting. Investigate and recommend next steps."
);Every .with() call turns on exactly one thing. No memory writes unless you called .withMemory(). No guardrail scanning unless you asked for it. No hidden system prompt doing work you didn’t authorize.
The builder pattern makes the orchestration contract explicit. You can read the agent definition and know exactly what will happen when you call .run(). That matters when you’re debugging why an agent behaved unexpectedly in production.
Where frameworks hide state
State management is the most common source of production surprises. Frameworks store state in three places:
In-memory arrays: Fast, simple, breaks on restart. Fine for demos, unusable for long-running agents or multi-worker deployments.
Local disk: Survives restarts, doesn’t scale horizontally. Works for single-instance deployments but creates consistency problems when you add replicas.
External stores: Redis, Postgres, vector databases. Adds latency and operational complexity but lets you scale, resume conversations, and audit history.
Most frameworks default to in-memory state and offer external stores as plugins. The problem is that switching from in-memory to external storage isn’t just a config change. It changes your error handling (what happens when Redis is down?), your latency profile (every turn now waits for a network round trip), and your consistency guarantees (what if two workers update the same conversation?).
If the framework doesn’t expose where state lives, you can’t reason about these trade-offs until you hit them in production.
Debugging without visibility
When an agent misbehaves, you need to answer three questions:
- What context did the model see?
- What did the model return?
- What did the framework do with that return value?
Frameworks that log only the final answer make it impossible to answer question three. You can see the model’s output in your LLM provider’s dashboard, but you can’t see whether the framework retried a tool call, skipped a step, or silently swallowed an error.
The fix is structured logging at every decision point:
- Log the full context before each LLM call (or a hash if it’s too large)
- Log the raw model response before parsing
- Log every tool call with inputs, outputs, and execution time
- Log every retry, timeout, or error with enough context to reproduce it
Frameworks that don’t expose hooks for this logging force you to wrap every method or fork the codebase.
When to build your own harness
You should consider a custom orchestration harness when:
- You need to audit every tool call for compliance or security
- Your failure modes don’t match the framework’s retry logic
- You’re integrating with internal systems that don’t fit the framework’s tool abstraction
- You need sub-second latency and can’t afford framework overhead
- You’re running agents in constrained environments (edge, mobile, air-gapped)
You should stick with a framework when:
- You’re prototyping and need to move fast
- Your use case fits the framework’s assumptions (stateless, short-lived, low-stakes)
- You have more trust in the framework’s error handling than your own
- You need ecosystem integrations (vector stores, observability tools, model providers)
The middle ground is using a framework that exposes its internals. You get the convenience of pre-built components but retain the ability to override decisions when the defaults don’t fit.
Technical Verdict
Use an opaque framework if you’re building demos, internal tools, or low-stakes automations where debuggability matters less than speed. LangChain and Semantic Kernel get you running fast.
Use an explicit framework (like Reactive Agents or AutoGen) if you’re deploying to production, need to audit tool calls, or have failure modes that don’t match framework defaults. The builder pattern and opt-in features give you control without forcing you to write everything from scratch.
Build a custom harness if you’re integrating with legacy systems, running in constrained environments, or have compliance requirements that frameworks can’t meet. You’ll write more code, but you’ll own every failure mode.
The key question is whether you can debug the framework’s decisions when things go wrong. If the answer is no, you’re trusting a black box. If you can’t afford that trust, crack it open.