PipesHub: Permission-Aware Context Layer for Enterprise Agent Infrastructure

Agents need context. The hard part is not vector search or embeddings. The hard part is maintaining permission boundaries when an agent queries Slack, Drive, Jira, and GitHub simultaneously, then cites the results without leaking data across teams.

PipesHub builds a permission-aware search layer over 50+ enterprise connectors and exposes it through MCP servers, SDKs, and built-in agents. It ranked #1 of 39 systems on EnterpriseRAG-Bench, ahead of OpenAI File Search, Amazon Q, Azure AI Search, and Vertex AI Search. The architecture shows how enterprise search becomes agent infrastructure when you design for both humans and machines.

What PipesHub Does

PipesHub connects to Slack, Google Drive, Jira, Confluence, GitHub, Microsoft 365, and 47 other sources. It indexes content, builds a knowledge graph, and exposes three interfaces:

MCP server: Claude Desktop, Cline, and other MCP clients can search across connectors with permission enforcement.
SDKs: Python and TypeScript libraries for custom agent harnesses.
Built-in agents: Pre-configured agents for common workflows (search, summarization, citation).

The system runs self-hosted or on PipesHub Cloud. Self-hosted means Docker Compose with Postgres, Redis, and Qdrant. State lives in Postgres (metadata, permissions, sync logs), Redis (task queues, rate limits), and Qdrant (vector embeddings).

Permission Boundary Architecture

When an agent queries “show me the latest design specs,” PipesHub must:

Resolve the user’s identity across connectors (OAuth tokens, service accounts).
Filter results to documents the user can access in each source system.
Return citations that preserve the original permission context.

The permission layer works like this:

# Simplified permission check during search
def search_with_permissions(query: str, user_id: str, connectors: list[str]):
    # Fetch user's access tokens for each connector
    tokens = get_user_tokens(user_id, connectors)
    
    # Query each connector with user's credentials
    results = []
    for connector in connectors:
        client = get_connector_client(connector, tokens[connector])
        # Connector SDK enforces source system RBAC
        docs = client.search(query, enforce_permissions=True)
        results.extend(docs)
    
    # Hybrid search: vector similarity + keyword match + graph traversal
    ranked = hybrid_rank(results, query)
    
    # Preserve source URLs and permission metadata in citations
    return [
        {
            "content": doc.text,
            "source": doc.url,
            "permissions": doc.access_list,
            "connector": doc.connector_type
        }
        for doc in ranked
    ]

The key: PipesHub does not merge permissions into a single access control list. Each result carries its source system’s permission metadata. When an agent cites a Jira ticket, the citation includes the Jira project key and the user’s access level in that project.

MCP Server vs. SDK vs. Built-In Agents

PipesHub exposes three interfaces. The differences matter for deployment shape and failure modes.

Interface	Use Case	State Management	Error Surface
MCP Server	Claude Desktop, Cline, MCP-compatible clients	Stateless per request, relies on client session	Client must handle rate limits, retries, token refresh
SDK	Custom agent harnesses, workflow orchestrators	Developer manages state, connection pools	Full control over retry logic, caching, fallback
Built-In Agents	Pre-configured workflows (search, summarize, cite)	PipesHub manages state, task queue, background sync	Opinionated flow, limited customization

The MCP server is a FastAPI app that implements the Model Context Protocol. It exposes tools like search_workspace, get_document, and list_connectors. When Claude Desktop calls search_workspace, the MCP server:

Authenticates the request (OAuth token or API key).
Calls the internal search service with the user’s permission context.
Returns results as structured JSON with citations.

The SDK gives you the same search service as a library. You handle connection pooling, retries, and state. Useful when you need to batch queries, cache results, or integrate with an existing orchestration layer (Temporal, Prefect, Airflow).

Built-in agents are opinionated workflows. The “Deep Research” agent chains multiple search calls, summarizes results, and generates a report. You configure the agent in the PipesHub UI, but you do not write code. The agent runs on PipesHub’s task queue (Celery + Redis).

Hybrid Search and Knowledge Graph

PipesHub combines three retrieval strategies:

Vector search: Qdrant embeddings for semantic similarity.
Keyword search: Postgres full-text search for exact matches.
Knowledge graph traversal: Neo4j-style graph (entities, relationships, metadata).

The knowledge graph is not a separate database. PipesHub builds it in Postgres using recursive CTEs and adjacency lists. Entities are documents, people, projects, and tags. Relationships are “mentions,” “authored by,” “linked to,” and “depends on.”

When you search “Q4 roadmap,” the hybrid ranker:

Finds documents with “Q4” and “roadmap” in the text (keyword).
Finds documents semantically similar to “quarterly planning” (vector).
Traverses the graph to find documents linked to the “Q4 Planning” project (graph).

Results are scored and merged. The graph traversal reduces false positives by surfacing documents that are contextually related but do not match keywords or embeddings.

Deployment Primitives

Self-hosted PipesHub runs in Docker Compose with five containers:

API server: FastAPI app (MCP server, REST API, webhooks).
Worker: Celery worker for background tasks (indexing, sync, agent execution).
Postgres: Metadata, permissions, sync logs, knowledge graph.
Redis: Task queue, rate limits, session cache.
Qdrant: Vector embeddings.

The API server is stateless. Workers pull tasks from Redis. Postgres holds all durable state. Qdrant is the only specialized dependency (you can swap it for Pinecone or Weaviate).

For production, you need:

Connector credentials: OAuth apps for Slack, Google, Microsoft, Atlassian. Each connector requires a separate OAuth flow.
Webhook endpoints: Slack, GitHub, and Jira send real-time updates. The API server must be publicly accessible or use a tunnel (ngrok, Cloudflare Tunnel).
Background sync: Workers poll connectors every 15 minutes (configurable). Rate limits vary by connector (Slack: 1 req/sec, Google Drive: 1000 req/100 sec).

PipesHub Cloud removes the OAuth and webhook setup. You connect connectors through the UI, and PipesHub manages credentials and sync.

Observability and Failure Modes

PipesHub logs every search query, connector sync, and agent execution to Postgres. The UI shows:

Query latency: P50, P95, P99 for search requests.
Connector health: Last sync time, error rate, rate limit hits.
Agent execution: Task duration, retry count, failure reason.

Common failure modes:

OAuth token expiration: Connectors fail silently when tokens expire. PipesHub retries with exponential backoff and sends a notification to the user.
Rate limit exhaustion: Slack and Google Drive rate limits are per-user, not per-app. If 100 users sync simultaneously, you hit limits. PipesHub queues sync tasks and spreads them over time.
Permission drift: A user loses access to a Jira project, but PipesHub’s cache still shows the document. The cache TTL is 5 minutes. After that, the document disappears from search results.
Graph staleness: The knowledge graph updates every 15 minutes. If a document is deleted in Slack, it remains in the graph until the next sync. This is acceptable for most workflows but breaks real-time use cases.

EnterpriseRAG-Bench Results

PipesHub ranked #1 of 39 systems on EnterpriseRAG-Bench, a benchmark for enterprise search and RAG systems. The benchmark tests:

Retrieval accuracy: Precision, recall, and F1 for document retrieval.
Citation quality: Do citations point to the correct source?
Permission enforcement: Does the system leak documents across users?

PipesHub scored 0.89 F1 (retrieval), 0.92 citation accuracy, and 100% permission enforcement. OpenAI File Search scored 0.81 F1, 0.85 citation accuracy, and 95% permission enforcement (it leaked documents when users shared folders).

The benchmark does not test latency, cost, or deployment complexity. PipesHub’s hybrid search is slower than vector-only systems (200ms vs. 50ms per query). The knowledge graph adds 50ms overhead. For interactive agents, this is acceptable. For batch workflows, you want caching.

Technical Verdict

Use PipesHub when:

You need permission-aware search across multiple enterprise sources.
You want to expose context to agents through MCP, SDKs, or pre-built workflows.
You can tolerate 200ms query latency and 15-minute sync intervals.
You have the infrastructure to run Postgres, Redis, and Qdrant (or you use PipesHub Cloud).

Avoid PipesHub when:

You need real-time search (sub-50ms latency, instant updates).
Your connectors are not in the 50+ supported list (you will write custom connectors).
You need multi-tenant isolation with strict data residency (self-hosted requires one deployment per tenant).
Your agents need stateful, long-running workflows (PipesHub agents are task-based, not session-based).

The hybrid search and knowledge graph architecture is the standout feature. Most enterprise search systems use vector-only retrieval and bolt on permissions as an afterthought. PipesHub treats permissions as a first-class constraint and uses graph traversal to reduce false positives. The MCP server makes it easy to connect agents, but the SDK gives you more control when you need it.

Agentic AI, minus the noise.