OpenAI just moved the orchestration layer from your codebase to their infrastructure. The Agents API is a managed service that handles session persistence, tool coordination, and long-running workflows without exposing raw checkpoint management. This is not a new model. It is a hosting primitive for cloud agents, and the design choices reveal how a platform vendor thinks about state, isolation, and failure recovery when the control loop lives on their servers instead of yours.
What the Codex Harness Abstracts
The Agents API is powered by something OpenAI calls the Codex harness. This is the orchestration engine that manages:
- Session lifecycle: Agents can run for hours or days without the client holding a connection open.
- Tool execution boundaries: The harness coordinates when to call external APIs, when to wait, and when to resume reasoning.
- State persistence: Conversation history, intermediate results, and tool outputs are stored server-side.
You submit a task. The harness decides when to invoke tools, when to prompt the model, and when to return control. You poll for results or register a webhook. The agent session persists independently of your client connection.
This is a departure from client-side agent loops where you manage the orchestration logic, handle retries, and persist state to your own database. OpenAI is betting that most developers would rather pay for managed orchestration than build it themselves.
Session Persistence Without Checkpoint Exposure
Long-running sessions are a first-class primitive in this API. That means the service must solve:
- Durable state storage: Conversation context, tool call history, and intermediate reasoning steps must survive server restarts.
- Session resumption: If a workflow spans multiple days, the agent must pick up where it left off without replaying the entire history.
- Cost attribution: Partial completions must be billed correctly, even if the session fails midway.
The harness likely uses a combination of append-only logs and snapshot checkpoints. Each tool call and model invocation gets written to durable storage. When you query session status, the API reconstructs the current state from the log. If the session crashes, the harness replays from the last checkpoint.
You do not see the checkpoint format. You do not manage the replay logic. You get a session ID and a status endpoint. The trade-off is control for convenience.
Tool Execution Isolation
When an agent calls a tool, the harness must enforce boundaries:
- Credential scoping: Tools should only access resources the session is authorized to touch.
- Timeout enforcement: A misbehaving tool cannot block the orchestration loop indefinitely.
- Cross-session isolation: One agent’s tool call should not contaminate another session’s state.
OpenAI likely runs tool execution in sandboxed environments with short-lived credentials. Each tool call gets a fresh execution context. The harness passes inputs, waits for outputs, and logs the result. If the tool times out, the harness records a failure and decides whether to retry or escalate to the model.
This is similar to how AWS Step Functions isolates Lambda invocations. The orchestration layer treats tool calls as black boxes with defined input/output contracts. The harness does not trust tool code to behave correctly.
Failure Recovery and Partial Completions
A managed orchestration service must answer: what happens when an agent workflow fails halfway through?
| Failure Mode | Likely Behavior | Cost Implication |
|---|---|---|
| Model timeout | Retry with exponential backoff | Billed for failed attempts |
| Tool call failure | Log error, prompt model to decide next step | Billed for orchestration overhead |
| Session expiration | Terminate and return partial results | Billed for completed steps only |
| Client disconnect | Session continues running, results available via polling | Full workflow billed regardless of client state |
The harness must log every step so you can audit what happened. If a tool call fails, the model might decide to retry with different parameters, skip the step, or abort the workflow. The orchestration layer does not make that decision. It defers to the model’s reasoning.
This is expensive. Every retry, every error handling branch, and every decision point consumes tokens. Managed orchestration shifts the cost from engineering time to API spend.
Architecture Shape
Here is what a typical interaction looks like:
import openai
# Create a long-running agent session
session = openai.agents.create(
instructions="Analyze this dataset and generate a report",
tools=[
{"type": "code_interpreter"},
{"type": "function", "function": {"name": "fetch_data", "description": "..."}}
],
metadata={"user_id": "12345"}
)
# The session runs asynchronously
session_id = session.id
# Poll for status
while True:
status = openai.agents.retrieve(session_id)
if status.state in ["completed", "failed"]:
break
time.sleep(10)
# Retrieve results
result = openai.agents.messages.list(session_id)The client does not hold the connection open. The harness manages the orchestration loop. You poll for updates or register a webhook to get notified when the session completes.
This is fundamentally different from streaming chat completions. The session is stateful, durable, and independent of the client lifecycle.
Observability Gaps
Managed orchestration introduces new observability challenges:
- Opaque decision trees: You see tool calls and model responses, but not the internal reasoning about when to invoke tools.
- Cost attribution: Partial completions and retries make it hard to predict spend per session.
- Debugging: If a session fails, you get logs, but not the ability to step through the orchestration logic.
You will need to instrument your tool implementations separately. The harness logs what it does, but not why. If an agent makes a surprising decision, you have to infer the reasoning from the conversation history.
Security Boundaries
The harness must enforce:
- Credential isolation: Each session gets scoped credentials that expire when the session ends.
- Tool sandboxing: Code execution happens in ephemeral containers with no network access except to approved endpoints.
- Audit logging: Every tool call, model invocation, and state transition gets logged for compliance.
You do not control the sandbox configuration. You trust OpenAI’s isolation guarantees. If you need custom security policies (e.g., running tools in your own VPC), you cannot use this service.
When to Use This
The Agents API makes sense when:
- You want to prototype agent workflows without building orchestration infrastructure.
- Your agents need to run for hours or days without client supervision.
- You trust OpenAI’s security boundaries and are comfortable with opaque orchestration logic.
- You prefer paying for managed services over hiring engineers to maintain state machines.
When to Avoid This
Skip this service if:
- You need fine-grained control over orchestration decisions (e.g., custom retry logic, circuit breakers).
- Your compliance requirements demand on-premises execution or custom audit trails.
- You want to minimize API costs by running orchestration logic locally.
- You need to integrate with tools that cannot be exposed via public APIs.
Technical Verdict
OpenAI’s Agents API is a convenience layer for developers who want to skip the plumbing. The Codex harness handles session persistence, tool coordination, and failure recovery, but you lose visibility into orchestration decisions and control over execution boundaries. This is a good fit for rapid prototyping and workflows where the cost of managed infrastructure is lower than the cost of building it yourself. If you need custom security policies, fine-grained observability, or hybrid cloud execution, you will still need to run your own orchestration layer.