mech.app

The mech.app newsletter

Agentic AI, minus the noise.

Get practical field notes on AI agents, automation, developer tools and security delivered to your inbox.

No spam. Unsubscribe anytime.

Automation

Omnara's Durable Agent API: How Postgres-Backed State Keeps Long-Running Agents Alive Across Restarts

Postgres-backed state persistence for durable agents: loop checkpoints, sandbox lifecycle, RBAC, and crash recovery architecture.

Source: github.com
Omnara's Durable Agent API: How Postgres-Backed State Keeps Long-Running Agents Alive Across Restarts

Most agent frameworks treat state as ephemeral. If the process dies, the agent forgets everything. Omnara takes the opposite approach: every agent loop, tool call, and sandbox session lives in Postgres. When the container crashes or you deploy a new version, the agent picks up where it left off.

This is not a framework for building agents. It is an API for running them durably. You send an HTTP request to create an agent, Omnara spins up the loop, provisions sandboxes, manages secrets, and writes every state transition to Postgres. The agent survives restarts, version upgrades, and infrastructure failures.

What Durable Means in Practice

Durable does not mean “logs everything.” It means the agent can resume mid-execution after an unexpected stop. That requires three things:

  • Loop state checkpoints: which step the agent is on, what tool calls have completed, what the next action should be.
  • Sandbox lifecycle tracking: which containers are alive, what files exist, what environment variables are set.
  • RBAC and secret bindings: which credentials the agent has access to, what permissions apply, what scopes are active.

When the agent process dies, Omnara reads the last checkpoint from Postgres, reconnects to the sandbox (or spins up a new one), reloads secrets, and continues. The agent does not start over. It does not lose context. It resumes.

Postgres Schema Design for Agent State

The schema separates three concerns: agent metadata, execution state, and resource bindings.

Agent metadata includes the agent’s ID, creation time, configuration (model, temperature, system prompt), and current status (running, paused, crashed). This lives in an agents table.

Execution state tracks the loop. Each iteration writes a row to an agent_steps table: timestamp, tool call request, tool call response, LLM output, next action. If the agent crashes, Omnara reads the last step and reconstructs the loop state.

Resource bindings link agents to sandboxes, secrets, and permissions. A sandboxes table tracks container IDs, IP addresses, and lifecycle events (created, running, stopped). A secrets table stores encrypted credentials. A permissions table maps agents to allowed actions (read file, execute command, call external API).

When an agent resumes, Omnara:

  1. Queries agent_steps for the last completed step.
  2. Checks sandboxes to see if the container is still alive.
  3. Reloads secrets and permissions from their tables.
  4. Continues the loop from the next action.

API Boundary and Control Flow

Omnara exposes five endpoints:

  • POST /agents: create a new agent with a system prompt, model, and tool list.
  • GET /agents/{id}: retrieve current state (running, paused, crashed).
  • POST /agents/{id}/pause: stop the loop without losing state.
  • POST /agents/{id}/resume: restart from the last checkpoint.
  • DELETE /agents/{id}: tear down the agent and clean up resources.

The agent loop runs in a separate process, not in the HTTP handler. When you call POST /agents, Omnara writes the agent config to Postgres and enqueues a task. A worker picks up the task, starts the loop, and begins writing checkpoints. The HTTP response returns immediately with the agent ID.

This separation matters for durability. If the API server crashes, the agent keeps running. If the agent worker crashes, the API server can restart it from the last checkpoint.

Sandbox Lifecycle and Crash Recovery

Sandboxes are Docker containers with a fixed set of tools (shell, Python, file system access). When an agent needs to run a command, Omnara:

  1. Checks if a sandbox already exists for this agent.
  2. If not, spins up a new container and writes the container ID to Postgres.
  3. Executes the command via Docker exec.
  4. Writes the output to agent_steps.

If the agent crashes mid-command, the sandbox stays alive. When the agent resumes, Omnara reconnects to the same container. Files, environment variables, and running processes persist.

If the sandbox crashes, Omnara detects the failure (via Docker health checks) and spins up a new container. The agent loses in-memory state but can retry the command. This is a trade-off: sandboxes are stateful but not durable. Postgres stores the execution history, not the sandbox filesystem.

RBAC and Secret Management

Every agent has a permission scope. The scope defines:

  • Which secrets the agent can access (API keys, database credentials).
  • Which external services the agent can call (Slack, GitHub, AWS).
  • Which file paths the agent can read or write.

Permissions are checked at the API boundary, not inside the agent loop. When the agent requests a tool call, Omnara validates the request against the permission table. If the agent tries to read a secret it does not have access to, the request fails before the LLM sees it.

Secrets are encrypted at rest using a key stored in environment variables. When the agent needs a secret, Omnara decrypts it, injects it into the tool call, and never writes the plaintext to Postgres. The agent_steps table logs that a secret was used, but not the value.

Failure Modes and Recovery Strategies

Failure TypeDetectionRecovery StrategyData Loss
Agent worker crashHealth check timeoutRestart from last checkpoint in agent_stepsNone (loop resumes)
Sandbox crashDocker health checkSpin up new container, retry last commandIn-memory sandbox state
Postgres connection lossQuery timeoutRetry with exponential backoff, pause agent if persistentNone (writes queued)
API server crashLoad balancer health checkAgents continue running, API requests queuedNone (agents independent)
Corrupt checkpointSchema validationRollback to previous step, mark agent as crashedLast step only

The hardest failure mode is a corrupt checkpoint. If the agent_steps table has invalid JSON or a missing field, Omnara cannot reconstruct the loop state. The current strategy is to rollback to the previous step and mark the agent as crashed. A human operator must inspect the logs and decide whether to resume or restart.

Deployment Shape

Omnara runs as three components:

  • API server: handles HTTP requests, writes to Postgres, enqueues tasks.
  • Agent workers: pull tasks from the queue, run agent loops, write checkpoints.
  • Postgres: stores all state (agents, steps, sandboxes, secrets, permissions).

The API server and workers are stateless. You can scale them horizontally. Postgres is the only stateful component. If you lose Postgres, you lose all agent state.

For production, run Postgres with replication (primary + standby). Use connection pooling (PgBouncer) to handle high write throughput. Use a message queue (Redis, RabbitMQ) between the API server and workers to decouple request handling from agent execution.

Code Example: Resuming an Agent from Postgres

def resume_agent(agent_id: str, db: Database):
    # Load agent metadata
    agent = db.query("SELECT * FROM agents WHERE id = ?", agent_id)
    if agent.status == "crashed":
        raise ValueError("Cannot resume crashed agent")
    
    # Load last checkpoint
    last_step = db.query(
        "SELECT * FROM agent_steps WHERE agent_id = ? ORDER BY timestamp DESC LIMIT 1",
        agent_id
    )
    
    # Reconnect to sandbox
    sandbox = db.query("SELECT * FROM sandboxes WHERE agent_id = ?", agent_id)
    if sandbox and docker.is_running(sandbox.container_id):
        container = docker.attach(sandbox.container_id)
    else:
        container = docker.create_sandbox(agent_id)
        db.execute("INSERT INTO sandboxes (agent_id, container_id) VALUES (?, ?)", 
                   agent_id, container.id)
    
    # Reload secrets and permissions
    secrets = db.query("SELECT * FROM secrets WHERE agent_id = ?", agent_id)
    permissions = db.query("SELECT * FROM permissions WHERE agent_id = ?", agent_id)
    
    # Continue loop from next action
    next_action = last_step.next_action
    run_agent_loop(agent, next_action, container, secrets, permissions, db)

This is pseudocode, but it shows the recovery flow: load metadata, find the last checkpoint, reconnect to the sandbox, reload secrets, and continue.

When Durable State Matters

Use Omnara when:

  • Agents run for hours or days (data analysis, report generation, multi-step workflows).
  • You need to deploy new code without killing in-flight agents.
  • Crash recovery is more important than raw throughput.
  • You already run Postgres and want to avoid adding a new state store.

Avoid Omnara when:

  • Agents complete in seconds (simple Q&A, single tool calls).
  • You need sub-second latency (high-frequency trading, real-time chat).
  • You want to build agents from scratch (Omnara is an API, not a framework).
  • You cannot tolerate Postgres as a single point of failure.

Technical Verdict

Omnara solves the durability problem by treating Postgres as the source of truth for agent state. This is a pragmatic choice: Postgres is well understood, widely deployed, and good enough for most agent workloads. The trade-off is write latency. Every loop iteration writes to Postgres, which adds 5-10ms per step. For long-running agents, this is acceptable. For high-frequency agents, it is not.

The architecture is simple: API server, workers, Postgres. No custom state store, no distributed consensus, no event sourcing. If you need durable agents and already run Postgres, Omnara is a solid starting point. If you need sub-second recovery or multi-region replication, you will need to extend the schema and add more infrastructure.

The open-source release (3.2k stars, 247 forks) suggests demand for this pattern. Most agent frameworks ignore durability. Omnara makes it the default.