Multi-Agent Firewall Architecture: How Privacy-Focused Proxies Sanitize LLM Interactions Without Breaking Tool Chains

LLMs handle sensitive data without built-in privacy controls. Every prompt you send to ChatGPT, Claude, or an internal RAG system leaves your network and gets logged by the provider. The standard mitigation is model-level guardrails or prompt filters, but those run after the data has already left your perimeter. A new ArXiv paper (2607.08282v1) proposes a different primitive: a privacy firewall that sits between users and LLMs, intercepting and sanitizing traffic before it reaches the model.

The architecture combines a browser extension for web traffic and a proxy for API calls, creating a unified interception layer for both synchronous requests and streaming WebSocket responses. The core innovation is a multi-agent pipeline that detects and redacts PII, credentials, and proprietary code without breaking multi-turn conversations or tool call sequences.

The Interception Layer

Most LLM privacy solutions assume you control the model or the application. This firewall assumes you control neither. It intercepts traffic at the network boundary using two components:

Browser extension: Hooks into web-based chat interfaces (ChatGPT, Claude, internal tools) and rewrites outgoing messages before they hit the network stack. Handles both HTTP(S) POST requests and WebSocket streams.

Proxy server: Sits between your application code and external LLM APIs. Intercepts programmatic calls (OpenAI SDK, LangChain, custom agents) and applies the same sanitization pipeline.

Both components feed into the same multi-agent detection system, which means you get consistent privacy enforcement across chat UIs and agentic workflows.

The interception happens at the transport layer, not the application layer. This matters because:

You don’t need to modify existing applications or retrain users
It works with closed-source LLM providers
It catches data leakage from third-party libraries and dependencies
It can enforce policy even when developers bypass official SDKs

Multi-Agent Detection Pipeline

The paper describes a hybrid detection approach: fast deterministic checks for known patterns, followed by LLM-driven semantic analysis for context-dependent leakage.

Stage 1: Deterministic detectors

Regex patterns for emails, phone numbers, credit cards, API keys
Named entity recognition for person names, organizations, locations
Code pattern matching for proprietary function signatures and internal URLs
Dictionary lookups for known sensitive terms (project codenames, internal systems)

Stage 2: Semantic analysis

When deterministic checks flag potential leakage, a local LLM evaluates context. Example: “John” in “John Doe” gets redacted, but “John” in “read from stdin” does not. The semantic agent considers:

Surrounding tokens and sentence structure
Domain-specific context (medical records vs. code comments)
User intent signals (asking for help with a bug vs. sharing customer data)

Stage 3: Code leakage prevention

Separate agent scans for proprietary code patterns:

Internal library imports and package names
Database schema references
Infrastructure hostnames and IP ranges
Commented-out credentials or config snippets

The paper reports achieving F1 scores of up to 94.93% on optimal configurations, which means roughly 5% false positive or false negative rate. In production, you tune this by adjusting which detectors run and how aggressive the semantic analysis is.

Redaction Without Breaking Context

The hard part is not detecting sensitive data. The hard part is removing it without destroying the conversation flow or breaking tool calls.

Token-level redaction: Instead of replacing “john.doe@company.com” with “[REDACTED]”, the firewall uses typed placeholders: “[EMAIL_1]”. This preserves:

Sentence structure for the LLM to parse
Reference continuity across multi-turn conversations
Tool call parameters that expect specific data types

State management: The firewall maintains a session-scoped mapping of placeholders to original values. When the LLM responds with “[EMAIL_1]”, the firewall reverse-maps it before showing the response to the user.

Tool call preservation: If an agent calls send_email(to="[EMAIL_1]", subject="..."), the firewall:

Detects the tool call in the LLM response
Checks if the placeholder maps to a user-approved contact
Either substitutes the real email and executes, or blocks and prompts the user

This means tool chains continue to work, but the LLM never sees the actual email address.

Deployment Shapes

The paper describes a layered architecture that enables deployment across heterogeneous environments, allowing organizations to balance computational cost, detection depth, and latency. Three deployment modes emerge from this flexibility:

Mode	Detection Agents	Privacy Boundary	Trade-offs
Local-only	Regex + local NER	All data stays on-premise	Lower detection accuracy, no cloud dependency
Hybrid	Local regex + cloud semantic LLM	Flagged snippets sent to trusted LLM	Better accuracy, partial cloud exposure
Cloud-assisted	Cloud-based detection pipeline	Pre-sanitized data sent to detection service	Fastest deployment, full cloud dependency

Local-only works for regulated industries (healthcare, finance) where data cannot leave the network. You run the entire pipeline on a local server or edge device. Detection quality depends on your local NER models and regex coverage.

Hybrid sends only flagged snippets to a cloud-based semantic analyzer. Example: deterministic checks catch “SSN: 123-45-6789” locally, but “my social is one two three…” gets sent to GPT-4 for semantic confirmation. This reduces cloud costs and exposure while improving detection accuracy.

Cloud-assisted offloads the entire pipeline to a managed service. Fastest to deploy, but you’re trusting a third party with pre-sanitized (but not fully anonymized) data.

Operational Considerations

The firewall exposes metrics for:

Detection rate by category (PII, credentials, code)
False positive rate (user overrides)
Latency impact on request processing
Redaction coverage (what percentage of prompts get modified)

Common failure modes to plan for:

Bypass via encoding: Users can base64-encode sensitive data to evade regex detectors. Mitigation: decode common encodings before scanning, or flag high-entropy strings for semantic analysis.

Context collapse: Aggressive redaction can make prompts unintelligible. Example: “Can you help me debug [CODE_1] when it connects to [URL_1] using [API_KEY_1]?” The LLM has no context. Mitigation: preserve non-sensitive context, or prompt users to rephrase.

Tool call ambiguity: If a tool expects user_id and the firewall redacts it, the tool fails. Mitigation: whitelist known-safe tool parameters, or require user confirmation before substituting placeholders.

Latency accumulation: Multi-stage detection adds processing overhead. For streaming responses, this can break the real-time feel. Mitigation: run deterministic checks synchronously, queue semantic analysis asynchronously, and only block if high-confidence leakage is detected.

Implementation Pattern

Here’s what the proxy interception pattern looks like for a programmatic API call (illustrative pseudocode based on the paper’s architecture):

class LLMFirewallProxy:
    def __init__(self, detectors: List[Detector], redactor: Redactor):
        self.detectors = detectors
        self.redactor = redactor
        self.session_state: Dict[str, Dict] = {}

    def intercept_request(self, request: HTTPRequest) -> HTTPRequest:
        # Only intercept LLM API endpoints
        if not self._is_llm_endpoint(request.host):
            return request

        session_id = self._extract_session(request)
        prompt = self._extract_prompt(request.body)
        
        # Run detection pipeline
        findings = []
        for detector in self.detectors:
            findings.extend(detector.scan(prompt))
        
        # Redact and store mappings
        redacted_prompt, mappings = self.redactor.apply(prompt, findings)
        self.session_state[session_id] = mappings
        
        # Rewrite request with redacted prompt
        request.body = self._inject_prompt(request.body, redacted_prompt)
        return request

    def intercept_response(self, response: HTTPResponse, session_id: str) -> HTTPResponse:
        if session_id not in self.session_state:
            return response
        
        # Reverse-map placeholders in LLM response
        response_text = self._extract_response(response.body)
        restored_text = self.redactor.restore(
            response_text, 
            self.session_state[session_id]
        )
        
        response.body = self._inject_response(response.body, restored_text)
        return response

The browser extension follows the same pattern but hooks into XMLHttpRequest and WebSocket APIs instead of using a proxy server.

When to Use This Architecture

Good fit:

You use third-party LLMs (OpenAI, Anthropic) and cannot control their logging
You have compliance requirements (GDPR, HIPAA) that prohibit sending PII to cloud services
You run multi-agent systems where tool calls might leak credentials or internal URLs
You want defense-in-depth: even if a developer accidentally logs sensitive data, the firewall catches it

Poor fit:

You control the LLM and can enforce privacy at the model level
Your agents need to reason about actual PII (fraud detection, personalized recommendations)
Latency is critical and you cannot tolerate additional processing overhead
Your threat model assumes the firewall itself is compromised (then you need end-to-end encryption, not a proxy)

The architecture works best when you treat the LLM as untrusted infrastructure. If you already trust your LLM provider, model-level guardrails are simpler.

Technical Verdict

This firewall architecture solves a real problem: how to use powerful cloud LLMs without leaking sensitive data. The multi-agent detection pipeline is practical, the interception layer is deployable, and the redaction strategy preserves enough context to keep tool chains working.

The reported 94.93% F1 score is promising but suggests you will need human-in-the-loop review for edge cases, especially in the first few months. The processing overhead is acceptable for asynchronous workflows but may impact real-time chat experiences.

Use this when you need privacy guarantees that your LLM provider cannot or will not give you. Avoid it if you need minimal latency or if your agents must reason about the actual sensitive data (not placeholders). The sweet spot is regulated industries running agentic workflows where compliance risk outweighs the cost of running a local detection pipeline.