Agents get to 90% fast. The last 10% is where you dictate paragraphs into a chat window or spend twenty minutes explaining which button needs a hover state. Remarc treats that feedback loop as infrastructure: structured comments with preserved context, queryable by agents over MCP, and stateful enough to prevent duplicate work across sessions.
The Problem: Feedback as Unstructured Noise
When you give an agent feedback through chat, you lose:
- Selection context: “That sentence in the third paragraph” requires the agent to re-parse the entire document.
- Visual anchors: Screenshots in chat are blobs. You can circle something, but the agent has no structured reference to what you circled.
- Status tracking: Once the agent responds, you have no shared record of what’s resolved, what’s pending, or what got missed.
The result is a loop of clarifications, re-explanations, and duplicated work. The agent is good at parsing mess, but garbage in still means garbage out.
Architecture: Context Capture + MCP Session Store
Remarc runs locally on macOS and captures three types of feedback:
- Text selection in any Mac app (no accessibility API, no browser extension).
- Screenshot annotation with drawn shapes and text overlays.
- Web element comments tied to DOM selectors.
Each comment stores:
- Original context (selected text, screenshot region, or CSS selector).
- Status (
open,in_progress,resolved). - Session ID (groups related comments into a single feedback pass).
- Resolution summary (agent-written note when closing a comment).
The agent reads the session over MCP, processes comments in priority order, updates statuses, and writes resolution summaries back. The human sees which comments are done without re-asking.
MCP Integration: Bidirectional Feedback Protocol
Most MCP servers expose tools for agents to call. Remarc exposes a session store that agents can query and mutate. The protocol looks like this:
// Agent queries open comments in a session
const session = await mcp.call("remarc.getSession", {
sessionId: "abc123"
});
// Returns structured comments
{
comments: [
{
id: "c1",
type: "text_selection",
context: "The button should have a hover state",
selection: "button.primary",
status: "open",
timestamp: "2026-09-08T14:22:00Z"
},
{
id: "c2",
type: "screenshot",
context: "Missing padding here",
imageUrl: "file://...",
annotations: [{ shape: "circle", x: 120, y: 340 }],
status: "open"
}
]
}
// Agent updates status and leaves resolution note
await mcp.call("remarc.updateComment", {
commentId: "c1",
status: "resolved",
resolution: "Added :hover pseudo-class with 0.2s opacity transition"
});The agent decides which comments to tackle first based on:
- Dependency order (comments referencing earlier work).
- Type (text selections are faster than screenshot interpretations).
- Explicit priority tags if the human adds them.
Status tracking prevents the agent from re-processing resolved comments in future sessions.
Context Preservation Without Browser Extensions
Remarc captures text selection in any Mac app without hooking into accessibility APIs. It uses macOS’s native pasteboard monitoring: when you select text and trigger Remarc’s hotkey, it reads the pasteboard, stores the selection, and associates it with the active app’s window title and process ID.
For web elements, Remarc injects a lightweight content script (only when you explicitly activate it) that captures the CSS selector path and visible text. The comment stores both, so the agent can locate the element even if the page re-renders.
Screenshots are simpler: Remarc captures the region, stores annotations as vector shapes (not rasterized), and gives the agent both the image and the shape metadata. The agent can parse “there’s a red circle at (120, 340)” without running OCR or vision models on the full screenshot.
Trade-offs and Failure Modes
| Aspect | Remarc’s Choice | Trade-off |
|---|---|---|
| Context capture | Pasteboard + window metadata | Breaks if app doesn’t expose selected text to OS |
| Web element tracking | CSS selector + visible text | Fragile if DOM structure changes between sessions |
| Agent prioritization | Heuristic (dependency + type) | No explicit DAG; agent may pick suboptimal order |
| Session isolation | Local file store, no sync | Multi-machine workflows require manual export |
| MCP transport | Stdio (local process) | No remote agent support without proxy layer |
Failure modes:
- Stale selectors: If the web app refactors its HTML, CSS selectors break. Remarc stores visible text as a fallback, but the agent may need to ask for clarification.
- Ambiguous screenshots: If you circle three buttons, the agent has to guess which one you meant. Text annotations help, but they’re optional.
- Session drift: If the agent resolves comments out of order, later comments may reference outdated state. The agent should re-validate context before marking a comment resolved.
Deployment Shape
Remarc is a local-first macOS app with no account, no telemetry, and no cloud dependency. The MCP server runs as a child process of the app, communicating over stdio. Agents connect via the standard MCP client library.
For remote agents (e.g., cloud-hosted coding assistants), you need a proxy:
- Run an MCP-to-HTTP bridge on your local machine.
- Expose it over a tunnel (Tailscale, ngrok, or WireGuard).
- Agent calls the HTTP endpoint; bridge forwards to local MCP server.
This adds latency but preserves the local-first security model. Remarc never sends your comments or screenshots to a third-party service.
Observability: What the Agent Sees
Agents log every MCP call (session query, status update, resolution write) to a local JSON file. You can tail it to see:
- Which comments the agent read.
- Which it skipped (and why, if it logs reasoning).
- How long each resolution took.
- Whether it updated statuses correctly.
Remarc’s UI shows a timeline of agent actions per comment, so you can spot patterns like “agent always skips screenshot comments” or “agent marks things resolved without leaving summaries.”
When to Use Remarc
Use it when:
- You’re iterating on agent output and the feedback loop is slower than the agent’s execution time.
- You need to preserve context across multiple feedback passes (e.g., a design review with 30 comments).
- Your agent supports MCP and you want structured input instead of chat transcripts.
Avoid it when:
- You’re giving high-level strategic feedback (“rethink the architecture”). Remarc is for granular, actionable comments.
- Your agent doesn’t support MCP and you can’t add a client library.
- You need multi-user collaboration. Remarc is single-user, local-first.
Technical Verdict
Remarc solves a real problem: the last 10% of agent polish is bottlenecked by unstructured feedback. By treating feedback as queryable infrastructure, it turns a paragraph of Slack rambling into a session the agent can work through programmatically.
The MCP integration is clean. The local-first architecture avoids the “upload your codebase to our cloud” trap. The context capture is pragmatic (pasteboard + window metadata) rather than perfect (no accessibility API means some apps won’t work).
The main limitation is session isolation. If you’re working across multiple machines or collaborating with other humans, you’ll need to export and import sessions manually. For solo builders iterating on agent output, it’s a sharp tool.
Source Links
- Primary source: Remarc on GitHub
- Discussion: Show HN thread (128 points)