A backend team shipped a local-first coding agent that kept repository secrets on each developer laptop during ordinary weekday work. The agent answered small questions after the first load, yet weekend batch reviews stalled whenever those laptops slept or left the office network. A product manager asked for a remote fallback that would never ship .env files, private keys, or customer dumps into a hosted context window.
The useful design was not another model comparison. It was a job router that classified work before any tokens were spent.
The Problem: Every Prompt Is Not Equally Safe to Send Elsewhere
Local-first agents fail in predictable ways when every prompt is treated as equally safe to route. Interactive edits want low latency and a warm local runtime. Overnight refactors want a machine that stays awake without closing the lid. Secret material wants disk and memory that the team already controls. Public fixtures can travel without violating residency rules.
A router that inspects those signals keeps the local-first promise without pretending the laptop is always the right host.
Four Signals That Actually Change the Route
The proposed classifier uses four signals that a wrapper can compute before it calls any model endpoint. None of the signals requires a public leaderboard score, and none of them assumes a particular vendor or hardware generation.
- Secret residency. The flag is true when the prompt, tools, or retrieved chunks include credentials, customer records, or private repository material.
- Offline requirement. The flag is true when the job must finish without a network path, including airplane work and locked-down environments.
- Expected duration. The estimate comes from historical task measurements or a simple heuristic (interactive edits under 30 seconds, batch refactors over 5 minutes).
- Host availability. The flag is true when the local machine will remain awake and reachable for the expected duration.
Teams should replace the duration heuristic with measurements from their own laptops, because a cold local load and a warm local load are different jobs.
Routing Decision Table
| Signal Combination | Route | Rationale |
|---|---|---|
| Secret residency = true | Local only | Secrets never leave controlled disk and memory |
| Offline requirement = true | Local only | No network path available |
| Duration > 5 min, Host availability = false | Remote | Laptop will sleep before completion |
| Duration < 30 sec, Host availability = true | Local | Low latency, warm runtime already present |
| Secret residency = false, Duration > 5 min, Host availability = true | Local or Remote | Policy decision based on cost and observability |
The table exposes the security boundary: secret residency and offline requirement are hard constraints. Duration and host availability are optimization signals.
Architecture: Where the Router Sits
The router sits between the agent reasoning layer and the execution infrastructure. It does not decide what to do (that is the agent’s job). It decides where to do it.
┌─────────────────┐
│ Agent Planner │
└────────┬────────┘
│ Task + Context
▼
┌─────────────────┐
│ Job Classifier │ ◄── Inspect signals before dispatch
└────────┬────────┘
│
┌────┴────┐
▼ ▼
┌────────┐ ┌────────┐
│ Local │ │ Remote │
│ Runner │ │ Host │
└────────┘ └────────┘The classifier runs on the same machine as the agent planner. It does not send task context to a remote service for classification, because that would leak the very secrets it is trying to protect.
Implementation: Classifying Without Leaking Context
The classifier inspects the task payload locally. It scans for patterns that indicate secret material: environment variable assignments, private key headers, database connection strings, and customer identifiers. It does not send the payload to a remote model for classification.
import re
from dataclasses import dataclass
from typing import List
@dataclass
class TaskSignals:
has_secrets: bool
offline_required: bool
estimated_duration_sec: int
host_available: bool
SECRET_PATTERNS = [
r'(?i)(password|secret|api[_-]?key|token)\s*[:=]',
r'-----BEGIN (RSA |EC )?PRIVATE KEY-----',
r'postgres://.*:.*@',
r'customer_id|ssn|credit_card'
]
def classify_task(prompt: str, tools: List[str],
offline_mode: bool, host_awake_duration_sec: int) -> str:
"""
Returns 'local' or 'remote' based on task signals.
Runs entirely on the local machine.
"""
has_secrets = any(re.search(pattern, prompt) for pattern in SECRET_PATTERNS)
# Heuristic: interactive edits < 30s, batch jobs > 300s
estimated_duration = 20 if len(prompt) < 500 else 400
host_available = host_awake_duration_sec > estimated_duration
signals = TaskSignals(
has_secrets=has_secrets,
offline_required=offline_mode,
estimated_duration_sec=estimated_duration,
host_available=host_available
)
# Hard constraints first
if signals.has_secrets or signals.offline_required:
return 'local'
# Optimization signals
if signals.estimated_duration_sec > 300 and not signals.host_available:
return 'remote'
return 'local' # Default to local when signals are ambiguousThe regex patterns are illustrative. Production systems should use a secret scanner like detect-secrets or trufflehog and maintain a team-specific allowlist for false positives.
Waking Remote Hosts: The Credential Transfer Problem
When the router selects a remote host, it must transfer task context without transferring secrets. The simplest design is to send only the task description and tool names, then let the remote host retrieve its own copies of shared resources (public repositories, documentation, approved datasets).
If the remote host needs access to private resources, the transfer must use a credential that is scoped to the task and expires after completion. The local machine generates a short-lived token that grants read-only access to the specific repository or database table required for the job.
def dispatch_to_remote(task_description: str, tools: List[str]) -> str:
"""
Sends task to remote host without sending secrets.
Remote host retrieves its own copies of shared resources.
"""
scoped_token = generate_scoped_token(
resources=['repo:example/public-fixtures'],
permissions=['read'],
ttl_seconds=3600
)
payload = {
'task': task_description,
'tools': tools,
'resource_token': scoped_token
}
response = requests.post('https://remote-host/execute', json=payload)
return response.json()['result']The remote host should log the token usage and revoke it immediately after the task completes. If the task fails, the token should still expire within the TTL window.
Failure Modes: When the Router Becomes a Bottleneck
The router is a single point of failure. If it crashes, the agent cannot dispatch tasks. If it misclassifies, secrets leak or jobs fail.
Misclassification risks:
- False negative (secrets sent to remote). The regex patterns miss a new secret format. The remote host logs the payload. The secret is now in two places.
- False positive (public work kept local). The classifier flags a public fixture as secret. The laptop sleeps before the job finishes. The task fails.
Mitigation:
- Run the classifier twice with different pattern sets and fail closed (route to local) when the results disagree.
- Log every routing decision with a hash of the task payload. Audit the logs weekly for misclassifications.
- Provide a manual override flag that forces local execution regardless of signals.
Router unavailability:
If the router process crashes, the agent should fail the task rather than guess a route. A fallback to “always local” is safer than “always remote” because it keeps secrets on controlled hardware.
Observability: Metrics That Matter
The router should emit metrics that distinguish routing decisions from orchestration decisions:
router.decision.local(counter): Tasks routed to local execution.router.decision.remote(counter): Tasks routed to remote execution.router.signal.secrets_detected(counter): Tasks flagged for secret residency.router.signal.offline_required(counter): Tasks flagged for offline requirement.router.classification_duration_ms(histogram): Time spent classifying each task.router.misclassification(counter): Tasks that failed due to incorrect routing.
The classification duration should be under 10ms for interactive tasks. If it exceeds 100ms, the regex patterns are too complex or the payload is too large.
State Management: Where Agent Memory Fits
The router does not manage agent memory. It inspects the task payload that the agent planner has already assembled. If the agent planner includes conversation history or retrieved documents in the payload, the router scans those too.
The agent planner is responsible for deciding what context to include. The router is responsible for deciding where that context can safely travel.
This separation keeps the router simple. It does not need to understand the agent’s reasoning process. It only needs to understand the security and availability constraints of the execution environments.
When the Router Should Not Exist
If every task can run locally and the laptop never sleeps, the router is unnecessary overhead. If every task can run remotely and secrets are already in a hosted vault, the router is also unnecessary.
The router is useful when:
- Some tasks contain secrets and some do not.
- Some tasks finish in seconds and some take hours.
- The local machine is not always available.
If those conditions do not hold, skip the router and route everything to a single execution environment.
Technical Verdict
Use this pattern when:
- You have a local-first agent that sometimes needs remote execution.
- You have secrets that cannot leave developer machines.
- You have long-running tasks that outlive laptop uptime.
- You can measure task duration and host availability.
Avoid this pattern when:
- All tasks can run in the same environment.
- You cannot reliably detect secrets in task payloads.
- The classification overhead exceeds the routing benefit.
- You do not have a secure credential transfer mechanism for remote hosts.
The router is not a model. It is a security boundary that happens to use heuristics. Treat it like a firewall rule, not a machine learning classifier.