Most agent frameworks treat execution as ephemeral scripts. You run a loop, it calls tools, maybe it logs to a file, then it exits. ZenML’s new Kitaru project takes a different approach: wrap agents in the same durable infrastructure you’d use for batch ML pipelines. That means versioned artifacts, replayable execution graphs, and observability hooks borrowed from MLOps tooling.
Hamza Tahir, ZenML’s founder, discussed the project on the Practical AI podcast. The core idea is simple: if you can version a trained model and replay its training run, you should be able to version an agent’s state and replay its decision path. The gap between agent demos and production systems is mostly plumbing, and MLOps already solved half of it.
What an Agent Harness Actually Does
A harness is not a framework. It’s a runtime wrapper that sits between your agent code and the execution environment. Think of it as a sidecar that captures state transitions, tool invocations, and decision points without forcing you to rewrite your agent logic.
Core responsibilities:
- State capture: Snapshots agent memory, tool outputs, and intermediate reasoning steps
- Replay semantics: Reconstructs execution from a checkpoint, replaying tool calls or skipping them with cached results
- Observability hooks: Emits traces, logs, and metrics in a format your existing monitoring stack understands
- Artifact versioning: Treats prompts, tool schemas, and model weights as immutable artifacts with lineage
The harness doesn’t dictate your agent architecture. You can run ReAct loops, plan-and-execute patterns, or custom state machines. The harness just ensures you can inspect, debug, and reproduce what happened.
MLOps Primitives That Transfer to Agents
ZenML built its reputation on pipeline orchestration for batch ML. Kitaru ports three primitives to agent execution:
1. Pipeline Abstraction
In MLOps, a pipeline is a directed acyclic graph of steps. Each step consumes artifacts, runs code, and produces new artifacts. Agents break this model because they loop and branch based on runtime conditions.
Kitaru’s solution: treat each agent iteration as a pipeline step. The loop becomes a meta-pipeline that spawns child pipelines. You get the same artifact tracking and caching semantics, but the graph can grow dynamically.
2. Artifact Stores
Every tool call result, every LLM response, every state transition gets written to an artifact store. This is not just logging. Artifacts are immutable, versioned, and queryable. You can diff two agent runs to see where their decision paths diverged.
3. Caching and Replay
If you replay an agent run and the inputs haven’t changed, Kitaru can skip expensive tool calls and reuse cached results. This is critical for debugging. You don’t want to re-run a $50 API call just to test a downstream logic branch.
| MLOps Primitive | Batch ML Semantics | Agent Semantics | Kitaru Adaptation |
|---|---|---|---|
| Pipeline | Static DAG | Dynamic loop with branches | Meta-pipeline spawns child steps per iteration |
| Artifact | Model weights, datasets | Tool outputs, LLM responses, state snapshots | Immutable, versioned, diffable |
| Caching | Skip training step if inputs match | Skip tool call if context matches | Content-addressed cache with replay flag |
| Observability | Metrics, logs, lineage | Traces, decision logs, tool call spans | Unified trace format for existing APM tools |
Architecture: How Kitaru Wraps Agent Execution
Kitaru doesn’t replace your agent framework. It wraps it. You keep using LangChain, AutoGPT, or custom code. The harness intercepts execution at three points:
1. Initialization
The harness loads the agent’s configuration (model, tools, prompts) from a versioned artifact store. It sets up observability hooks and initializes the replay cache.
2. Execution Loop
Each iteration of the agent loop becomes a pipeline step. The harness captures:
- Input state (memory, context, previous tool results)
- LLM call (prompt, response, token count, latency)
- Tool invocations (parameters, outputs, errors)
- Output state (updated memory, next action)
3. Termination
The harness writes a final artifact containing the full execution trace, all intermediate states, and a replay manifest. You can load this artifact later to resume or debug.
from kitaru import AgentHarness
from your_agent import MyReActAgent
# Wrap your agent in the harness
harness = AgentHarness(
agent=MyReActAgent(),
artifact_store="s3://my-bucket/agents",
enable_replay=True,
cache_tool_calls=True
)
# Run the agent (harness captures everything)
result = harness.run(
task="Find the cheapest flight to Berlin",
max_iterations=10
)
# Later: replay from a checkpoint
harness.replay(
run_id="abc123",
from_step=5,
override_tool_results={"search_flights": cached_results}
)Replay Semantics: What Gets Snapshotted
Replay is not just re-running the same code. It’s reconstructing the exact decision path the agent took, then optionally diverging at a specific step.
What Kitaru snapshots:
- Prompts and responses: Full text, not just embeddings
- Tool call parameters: Exact arguments passed to each tool
- Tool outputs: Raw results, not just success/fail flags
- Agent memory: Vector store state, conversation history, working memory
- Model weights: If you’re fine-tuning, the exact checkpoint used
- Environment state: API versions, tool schemas, external service responses
Replay modes:
- Full replay: Re-execute everything, useful for validating determinism
- Cached replay: Skip tool calls, use stored results, fast debugging
- Divergent replay: Start from step N, override one input, see what changes
Observability: Traces, Not Just Logs
Kitaru emits OpenTelemetry traces. Each agent iteration is a span. Tool calls are child spans. LLM requests are nested spans with token counts and latency.
This plugs into your existing observability stack. If you’re running Jaeger, Honeycomb, or Datadog, you get agent traces alongside your API traces. You can correlate an agent’s decision with the downstream services it called.
Key metrics Kitaru tracks:
- Iterations per task (how many loops before termination)
- Tool call latency (which tools are slow)
- LLM token usage (cost per task)
- Cache hit rate (how often replay skips work)
- Error rates (which tools fail most often)
Failure Modes and Mitigation
Agent harnesses introduce new failure surfaces:
| Failure Mode | Symptom | Mitigation |
|---|---|---|
| Artifact store latency | Slow agent startup, high write latency | Use local cache with async upload |
| Replay divergence | Replayed run produces different results | Snapshot external API responses, not just tool outputs |
| State explosion | Artifact store fills with GB of traces | Prune old runs, compress snapshots, sample traces |
| Cache invalidation | Stale tool results break replay | Content-address cache keys, include tool version in hash |
The biggest risk is assuming replay will always be deterministic. If your agent calls a live API that returns different results, replay will diverge. Kitaru can snapshot API responses, but you need to decide whether that’s acceptable for your use case.
When to Use an Agent Harness
Good fit:
- You’re running agents in production and need to debug failures
- You want to version agent behavior like you version models
- You need to replay agent runs for compliance or auditing
- You’re building agent fleets and need centralized observability
Bad fit:
- You’re prototyping and don’t care about durability yet
- Your agents are stateless request-response systems (just use API logs)
- You can’t afford the storage overhead of full execution traces
- Your agents run in environments without persistent storage
Technical Verdict
Use Kitaru if you replay agent runs at least twice per week for debugging, your artifact storage budget is under $500 per month, or you need compliance audit trails showing exact decision paths. It’s a strong fit when agents run 10+ iterations per task and you’re already using ZenML or similar MLOps tooling. The harness overhead (5-15% latency, 2-10 GB storage per 100 runs) pays off when you can skip re-running expensive tool calls during debugging.
Avoid it if your agents are stateless request-response systems where standard API logging suffices. Skip the harness if you run agents in ephemeral containers without persistent storage, or if you’re still in prototype phase with fewer than three production agents. The replay infrastructure adds complexity that only matters when you’re debugging non-deterministic failures or need to reconstruct past decisions for auditing.
The decision hinges on replay frequency and storage cost. If you’re not looking at old agent runs to understand failures, you don’t need the harness. If you are, and you can afford 100-500 MB per agent run, Kitaru gives you the same debugging experience you’d expect from batch ML pipelines.