Most agent frameworks assume you have a persistent network connection, unlimited compute, and a process that won’t be killed by the OS. Mobile devices break all three assumptions. PalmClaw is an open-source framework that runs the entire agent loop (session management, memory, tool calls, and orchestration) natively on the phone, not in the cloud.
The paper (arXiv 2607.13027v1) reports an 11.5% improvement in task success and a 94.9% reduction in completion time compared to the strongest baseline. The speed gain comes from eliminating server round-trips. The success gain comes from exposing device capabilities as structured tools instead of simulating GUI taps and swipes.
Why GUI Automation Fails for Mobile Agents
Existing mobile agents mainly operate through graphical user interface actions: tap this button, swipe that list, type into this field. This approach creates three problems:
- Long action sequences. Opening an app, navigating menus, and filling forms can take dozens of steps. Each step depends on the previous one succeeding.
- Interface brittleness. UI layouts change between OS versions, device sizes, and app updates. A tap coordinate that works on one phone fails on another.
- Unclear execution boundaries. When does a “send email” task finish? After the tap? After the network call? After the confirmation toast?
PalmClaw replaces GUI simulation with native device tools. Instead of scripting taps to send an email, the agent calls a send_email tool with explicit arguments (recipient, subject, body) and receives a structured result (success boolean, message ID, error code).
| Approach | Steps per Task | Brittleness | Execution Clarity |
|---|---|---|---|
| GUI automation | 20+ steps, 5-10s total | Breaks on UI changes, device variations | Ambiguous (screen parsing required) |
| Native tools | 1-3 tool calls, 0.5-2s total | Stable (OS APIs versioned) | Explicit (typed results, error codes) |
| Hybrid | Variable (GUI for unsupported apps) | Moderate (fallback to GUI) | Mixed (clear for tools, ambiguous for GUI) |
Architecture: Orchestration Loop on the Device
The framework runs entirely on the mobile device. No cloud orchestrator, no server-side LLM calls for tool selection. The agent loop executes locally.
The following pseudocode illustrates the agent loop:
while not task_complete:
# LLM runs on-device (quantized model)
action = model.predict_next_action(context, available_tools)
# Tool execution happens in native code
result = device_tools.execute(action.tool_name, action.args)
# State persists to local storage after each step
state_manager.checkpoint(context, result)
# Update context for next iteration
context.append(result)The key architectural choices:
- On-device LLM inference. Uses quantized models (likely 4-bit or 8-bit) to fit within mobile memory constraints. Latency is higher than cloud inference, but you avoid network round-trips entirely.
- Native tool bindings. Device capabilities (camera, contacts, location, notifications) are exposed as tools with typed arguments and structured outputs. No screen parsing.
- Stateful checkpointing. After each tool call, the framework persists the conversation context and execution state to local storage. If the OS kills the process, the agent can resume from the last checkpoint.
Tool Protocol: Explicit Arguments and Execution Boundaries
PalmClaw exposes device capabilities as device tools with explicit arguments, structured results, and clearly defined execution boundaries. Each tool has:
- Typed input schema. Arguments are validated before execution. No ambiguous natural language instructions.
- Structured output. Results are JSON objects with success flags, data payloads, and error codes.
- Synchronous completion. The tool call blocks until the operation finishes or times out. The agent knows when the action is done.
Example tool definition (conceptual):
{
"name": "send_sms",
"description": "Send a text message to a phone number",
"parameters": {
"recipient": {"type": "string", "pattern": "^\\+?[0-9]{10,15}$"},
"message": {"type": "string", "maxLength": 1600}
},
"returns": {
"success": {"type": "boolean"},
"message_id": {"type": "string", "optional": true},
"error": {"type": "string", "optional": true}
}
}This structure eliminates the ambiguity of GUI-based actions. The agent doesn’t need to interpret whether a tap succeeded by analyzing pixels. It receives a boolean.
State Management Under OS Constraints
Mobile operating systems kill background processes aggressively to save battery. An agent that runs for 30 seconds might be killed three times. PalmClaw handles this with:
- Checkpoint-after-every-step. The framework writes the conversation history, tool results, and next-action plan to persistent storage after each tool execution.
- Resumption protocol. When the process restarts, the agent loads the last checkpoint, validates that the previous tool call completed, and continues from the next step.
- Idempotent tool design. Tools are designed to be safely retried. If the agent isn’t sure whether a tool call succeeded, it can re-execute without side effects (or the tool returns a “already done” status).
Checkpoint persistence must handle power loss and abrupt termination. Android implementations use DataStore or SharedPreferences for atomic writes. iOS uses FileManager with atomic operations via NSFileCoordinator. Both platforms write to a temporary file, then rename it to the final checkpoint path (an atomic operation at the filesystem level that ensures the checkpoint is never partially written), ensuring the checkpoint is never corrupted.
Security Boundaries: Permissions and Sandboxing
PalmClaw exposes device capabilities as tools with explicit arguments and structured results. Tools that access sensitive APIs (location, contacts, camera) require explicit user permission at install time or first use. The agent doesn’t have blanket access to all device capabilities. The framework exposes only the tools that the agent’s configuration explicitly declares.
The paper does not detail sandboxing mechanisms beyond standard app permissions, so the agent runs in the same security context as a normal app. Audit logs for tool calls (timestamp, arguments, result) allow users to review agent actions.
Latency and Battery Trade-Offs
The 94.9% latency reduction likely stems from eliminating multi-step GUI automation rather than per-step inference speedup. A GUI-based agent might need 20 taps to send an email. A tool-based agent needs one send_email call. The reduction is in task completion time, not individual inference latency.
On-device inference is slower than cloud inference per step, but you save the network round-trip. For tasks with multiple tool calls, the elimination of GUI navigation sequences provides the dominant speedup.
Battery impact is modest for short-lived tasks but compounds for long-running agents. Quantized model inference consumes power proportional to the number of forward passes and model size. Tasks that complete in under a minute have negligible battery impact compared to typical app usage.
Implementation Considerations
If you’re building a similar framework, these are practical constraints discovered during implementation:
- Model quantization is mandatory. A full-precision 7B model won’t fit in mobile RAM. Use 4-bit or 8-bit quantization. Expect accuracy drops, especially for complex reasoning.
- Context window limits hit faster. Mobile models typically support 2K-4K tokens, not 32K. Design tasks that fit within this budget or implement aggressive context pruning.
- Tool execution must be async-safe. The OS can suspend your process mid-execution. Use OS-provided async APIs (like Android WorkManager or iOS BackgroundTasks) to ensure tool calls complete even if the app is backgrounded.
- Test on low-end devices. A framework that works on a flagship phone might thrash on a 3-year-old mid-range device with 4GB RAM.
Failure Modes
Common failure modes include:
- Model hallucination on tool arguments. The on-device model might generate invalid JSON or incorrect argument types. Validate inputs before execution. A malformed tool call might look like
{"name": "John Doe", "phone":}with a missing value that breaks parsing. - Checkpoint corruption. OS kills process during checkpoint write, leaving partial state on disk. Atomic write operations (write to temp, rename) prevent this, but you must handle cases where the temp file exists on restart.
- Tool timeouts. Network-dependent tools (like sending email) might hang if connectivity is poor. Implement timeouts and retry logic.
- Permission denial. If the user revokes a permission mid-task, the agent must handle the failure gracefully and either request re-permission or abort the task.
- Process termination mid-checkpoint. If the OS kills the process between writing checkpoint metadata and writing the actual state blob, the agent restarts with inconsistent state. Use transactional storage APIs or version your checkpoints.
Technical Verdict
Use PalmClaw-style on-device orchestration when:
- You need to access device-specific capabilities (sensors, local apps, user data) without uploading everything to the cloud.
- Network latency or reliability is a problem (offline use cases, rural areas, airplanes).
- Privacy requirements prohibit sending user data to external servers.
- You’re building a consumer app where users expect instant responses and don’t want to wait for server round-trips.
Avoid it when:
- You need complex reasoning that requires large models (70B+). On-device inference won’t cut it.
- Your tasks involve long-running operations (multi-minute workflows). Battery drain becomes unacceptable.
- You need cross-device orchestration (agent starts on phone, continues on laptop). On-device state doesn’t sync automatically.
- You’re prototyping rapidly and don’t want to deal with mobile-specific constraints (quantization, checkpointing, OS lifecycle).
The framework makes sense for personal assistant use cases (schedule meetings, send messages, set reminders) where the agent operates on local data and completes tasks in seconds. It’s a poor fit for research agents that need to browse the web for 10 minutes or creative agents that need large context windows.
Source Links
- PalmClaw Paper (arXiv 2607.13027v1) – Full research paper with experimental results
- PDF – Direct PDF download