Most agent orchestrators spin up a container or VM per agent. That works for ten agents. It breaks at a thousand. Rivet (YC W23) replaces the sandbox-per-agent model with lightweight actors backed by S3. Each actor holds in-memory state for the duration of a turn, then checkpoints to object storage. The result is high-density deployment without the overhead of persistent compute.
This is not a workflow engine. It is a compute model designed for long-running, stateful agent turns that need to survive crashes and scale horizontally.
The Sandbox Problem
Traditional agent deployments use one of three patterns:
- Container per agent: High isolation, high cost. Each agent gets a full runtime. Memory overhead starts at 50 MB per container, often much higher.
- Shared runtime with process isolation: Lower overhead, but state management becomes complex. Crashes take down multiple agents.
- Serverless functions: Stateless by design. Every turn requires rehydrating context from external storage.
All three patterns struggle with long-running agent turns. An agent analyzing a codebase, debugging a deployment, or orchestrating API calls across minutes or hours needs persistent state without persistent compute cost.
Actor Model for Agents
Rivet uses the actor model. Each agent is an actor: a lightweight process with private state, a mailbox for messages, and a lifecycle independent of other actors.
Key properties:
- Single-threaded execution: No locks, no race conditions within an actor.
- Message-driven: Agents communicate via asynchronous messages, not shared memory.
- Location transparency: Actors can move between nodes without changing code.
- Supervision trees: Parent actors monitor children and restart them on failure.
In Rivet’s implementation, actors are not persistent processes. They spin up on demand, load state from S3, execute a turn, checkpoint state back to S3, and terminate. The actor abstraction persists; the OS process does not.
S3 as the State Layer
S3 (or any object store) serves as the durable state backend. When an actor starts:
- Fetch the latest checkpoint from S3 using a deterministic key (agent ID + turn counter).
- Deserialize state into memory.
- Execute the agent turn (tool calls, LLM interactions, state mutations).
- Serialize state and write a new checkpoint to S3.
- Terminate the process.
This pattern decouples compute from storage. Actors only consume CPU and memory during active turns. Between turns, state lives in S3 at pennies per GB per month.
State Checkpointing Flow
class AgentActor:
def __init__(self, agent_id: str, s3_client):
self.agent_id = agent_id
self.s3_client = s3_client
self.state = None
self.turn_counter = 0
def load_state(self):
key = f"agents/{self.agent_id}/checkpoint-{self.turn_counter}.json"
try:
obj = self.s3_client.get_object(Bucket="rivet-state", Key=key)
self.state = json.loads(obj['Body'].read())
except self.s3_client.exceptions.NoSuchKey:
self.state = {"context": [], "tools": []}
def execute_turn(self, message):
# Load previous state
self.load_state()
# Process message, call tools, update context
self.state["context"].append(message)
response = self.call_llm(self.state["context"])
self.state["context"].append(response)
# Checkpoint updated state
self.turn_counter += 1
self.save_state()
return response
def save_state(self):
key = f"agents/{self.agent_id}/checkpoint-{self.turn_counter}.json"
self.s3_client.put_object(
Bucket="rivet-state",
Key=key,
Body=json.dumps(self.state)
)Failure Modes and Recovery
Actors crash. Networks partition. S3 writes fail. The architecture must handle all three.
Mid-Turn Crash
If an actor crashes before checkpointing, the turn is lost. The next invocation loads the previous checkpoint and retries. This requires idempotency in tool calls or explicit turn IDs to detect duplicates.
Mitigation: Write a provisional checkpoint before executing expensive operations (multi-step tool chains, long LLM calls). If the actor crashes, the next invocation can resume from the provisional state.
S3 Write Failure
S3 writes can fail due to network issues, rate limits, or transient errors. The actor must retry with exponential backoff.
Mitigation: Use S3’s conditional writes (if-none-match) to prevent overwriting a newer checkpoint. Include a monotonic turn counter in the key to detect out-of-order writes.
Concurrent Requests
Two messages arrive for the same agent simultaneously. Without coordination, both actors load the same checkpoint, execute in parallel, and write conflicting state.
Mitigation: Use a distributed lock (DynamoDB conditional writes, Redis SETNX) to serialize access per agent. Only one actor can hold the lock for a given agent ID at a time.
Cost and Latency Trade-offs
| Dimension | Actor + S3 | Container per Agent | Serverless Functions |
|---|---|---|---|
| Idle cost | $0 (S3 storage only) | High (persistent compute) | $0 |
| Cold start | 50-200 ms (S3 fetch + deserialize) | N/A (always warm) | 100-500 ms (runtime init) |
| Memory density | 1000+ agents per node | 10-50 agents per node | N/A (managed) |
| State consistency | Eventual (S3 latency) | Immediate (in-memory) | Eventual (external store) |
| Failure recovery | Automatic (load last checkpoint) | Manual (restart container) | Automatic (retry function) |
The actor model wins on density and idle cost. It loses on cold start latency and state consistency. For agents with infrequent turns (minutes to hours between messages), the trade-off favors actors. For high-frequency, low-latency agents (sub-second turns), containers or shared runtimes perform better.
Comparison to Durable Execution Patterns
Rivet’s actor model resembles durable execution frameworks like Temporal, Restate, and DASP. All three use external storage to persist execution state and recover from failures.
Temporal: Workflow-centric. State is event-sourced. Requires a dedicated cluster (database + workers). Heavier operational footprint.
Restate: Function-centric. State is key-value. Uses RocksDB for local state and Kafka for replication. Optimized for low-latency, high-throughput services.
DASP: Session-centric. State is opaque blobs. Designed for interactive agents with long-lived sessions. No built-in orchestration.
Rivet: Actor-centric. State is S3 checkpoints. Optimized for high-density, low-frequency agent workloads. Minimal infrastructure (just S3 + compute nodes).
Rivet sits between DASP (too minimal) and Temporal (too heavy). It provides orchestration without requiring a dedicated cluster.
Deployment Shape
A Rivet deployment consists of:
- Compute nodes: Run actors on demand. Stateless. Scale horizontally.
- S3 bucket: Stores checkpoints. One bucket per environment.
- Message queue: Delivers messages to actors. SQS, Kafka, or Redis Streams.
- Lock service: Serializes access per agent. DynamoDB, Redis, or etcd.
Actors are scheduled onto compute nodes using a simple round-robin or least-loaded strategy. No complex placement logic. If a node dies, the scheduler routes messages to another node. The actor loads state from S3 and continues.
Observability
Actor-based systems are hard to debug. Messages flow asynchronously. State is distributed across S3 and memory. Failures are non-deterministic.
Tracing: Instrument message sends and receives. Use OpenTelemetry to track message flow across actors. Include agent ID and turn counter in trace context.
Logging: Log every checkpoint write. Include S3 key, turn counter, and state size. Use structured logs (JSON) for easy querying.
Metrics: Track actor start/stop rate, checkpoint latency, S3 read/write errors, and lock contention. Alert on high error rates or slow checkpoints.
State inspection: Provide a CLI or API to fetch the latest checkpoint for any agent. Essential for debugging stuck agents or corrupted state.
Security Boundaries
Actors share compute nodes. Without isolation, a malicious agent can read memory from other agents or exhaust node resources.
Process isolation: Run each actor in a separate OS process. Use seccomp or AppArmor to restrict syscalls.
Resource limits: Enforce CPU, memory, and I/O limits per actor. Use cgroups or container runtimes (gVisor, Firecracker).
S3 access control: Use IAM roles to restrict which agents can read/write which checkpoints. Prefix keys with agent ID and validate access on every read/write.
Network isolation: Agents should not communicate directly. All messages flow through the message queue. Use network policies to block inter-actor traffic.
When to Use Rivet
Use Rivet when:
- Agents have long, infrequent turns (minutes to hours between messages).
- You need high-density deployment (hundreds to thousands of agents per node).
- Idle cost matters more than cold start latency.
- State fits in memory (< 1 GB per agent).
- You already use S3 or compatible object storage.
Avoid Rivet when:
- Agents require sub-second turn latency.
- State is too large to checkpoint frequently (> 1 GB).
- You need strong consistency guarantees (immediate state visibility).
- You prefer managed orchestration (Temporal, Prefect, Airflow).
Technical Verdict
Rivet’s actor model is a pragmatic middle ground for agent orchestration. It avoids the overhead of persistent sandboxes without sacrificing state durability. The S3 checkpointing pattern is simple, cheap, and scales horizontally. The trade-off is cold start latency and eventual consistency.
For teams building agent platforms with hundreds of concurrent agents, Rivet’s density advantage is compelling. For teams building low-latency, high-frequency agents, the actor model adds unnecessary complexity. The architecture shines when idle cost dominates total cost of ownership.
The failure modes are well-understood (idempotency, retries, locks). The operational footprint is minimal (S3 + compute + queue). The observability story needs work (distributed tracing is hard), but the primitives are there.
If you are running agents in production and hitting the limits of container-per-agent deployments, Rivet is worth evaluating. If you are still prototyping or running fewer than 50 agents, stick with simpler patterns.