GitHub’s Aspire team shipped a production agentic workflow that closes the gap between merged product code and updated documentation. The system detects changes in product repositories, generates documentation pull requests in separate doc repositories, and routes them through subject matter expert (SME) review loops. This is not a prototype. It runs in production across GitHub’s own infrastructure.
The problem is familiar: product teams ship features faster than documentation teams can track them. By the time a feature lands in main, the corresponding docs are stale or missing. Manual tracking does not scale. GitHub’s solution orchestrates agents across repository boundaries to automate the entire pipeline from code merge to reviewed doc PR.
Architecture: Cross-Repo Event Flow
The workflow starts with a webhook trigger. When a PR merges in a product repository, the system evaluates whether the change requires documentation updates. This evaluation step uses an LLM to analyze commit messages, file diffs, and PR descriptions against a set of documentation relevance heuristics.
If the change qualifies, the agent clones the target documentation repository and generates a draft PR. The draft includes:
- Updated prose reflecting the product change
- Links back to the original product PR
- Metadata tagging the relevant SME for review
The agent does not commit directly to main. It opens a PR in the docs repository and assigns it to the appropriate reviewer based on a mapping file that lives in the product repo. This mapping file is the coordination contract between product and documentation teams.
State Management and Context Retention
The agent needs context about existing documentation structure, style guidelines, and product semantics. GitHub’s implementation uses a retrieval-augmented generation (RAG) pattern with three context sources:
- Documentation corpus index: A vector store of all existing docs, updated nightly
- Product change metadata: Commit history, issue links, and PR descriptions from the product repo
- Style guide and templates: Markdown templates and tone guidelines stored in the docs repo
When generating a doc update, the agent queries the vector store to find related existing content, then uses the style guide to match tone and structure. This prevents the agent from inventing new terminology or contradicting established patterns.
Context is passed between steps using GitHub Actions artifacts. Each workflow run stores intermediate outputs (analysis results, draft content, review assignments) as JSON artifacts that subsequent steps can load. This keeps the workflow stateless at the orchestration level while preserving necessary context across steps.
SME Review Loop Integration
The human-in-the-loop design is critical. The agent does not auto-merge documentation. Instead, it creates a PR and waits for SME approval. The SME can:
- Approve and merge
- Request changes with inline comments
- Reject and close
If the SME requests changes, the agent does not automatically regenerate content. The workflow treats the PR like any other: the SME edits directly or provides feedback for a human writer to incorporate. This avoids infinite revision loops where the agent and SME argue over phrasing.
The system tracks review latency and sends reminders if a doc PR sits unreviewed for more than three business days. These reminders go to a Slack channel monitored by the documentation team lead.
Credential and Access Control
Operating across repositories requires careful permission management. The agent runs under a dedicated GitHub App with scoped permissions:
| Permission Scope | Access Level | Reason |
|---|---|---|
| Product repo (read) | Contents: read, Pull requests: read | Fetch PR metadata and diffs |
| Docs repo (write) | Contents: write, Pull requests: write | Create branches and open PRs |
| Org members (read) | Members: read | Resolve SME usernames from mapping file |
| Checks (write) | Checks: write | Post status updates on doc PR creation |
The GitHub App uses installation tokens that expire after one hour. Each workflow run requests a fresh token at the start. This limits the blast radius if credentials leak.
The agent does not have access to private repositories outside the Aspire team’s scope. Cross-org documentation updates are not supported. The system is designed for internal use within a single GitHub organization.
Observability and Failure Modes
GitHub’s implementation logs every step to a structured logging pipeline that feeds into their internal observability stack. Key metrics:
- Detection accuracy: Percentage of merged PRs correctly flagged as requiring doc updates
- Draft quality: SME acceptance rate on first submission
- Review latency: Time from doc PR creation to merge
- False positives: PRs flagged for docs that did not need updates
The most common failure mode is incorrect SME assignment. If the mapping file is outdated, the agent assigns the wrong reviewer. The fallback is to assign the docs team lead, who manually reassigns.
Another failure mode is context drift. If the documentation corpus index is stale, the agent may generate content that contradicts recently merged docs. The nightly index rebuild mitigates this, but there is still a window where the agent operates on yesterday’s snapshot.
The system does not retry failed doc PR creations automatically. If the agent cannot push a branch (due to permission errors or network issues), the workflow fails and alerts the on-call engineer. Retries are manual to avoid spamming the docs repo with duplicate branches.
Deployment Shape
The workflow runs as a GitHub Actions workflow in a dedicated .github/workflows directory within the product repository. Each product repo that opts into the system includes:
- A workflow YAML file that triggers on PR merge
- A
CODEOWNERS-style mapping file linking code paths to SMEs - A configuration file specifying the target docs repository
The agent itself is a Docker container hosted in GitHub Container Registry. The container includes:
- Python runtime with LangChain for orchestration
- OpenAI SDK for LLM calls (using Azure OpenAI endpoints)
- GitHub CLI for repository operations
- Jinja2 templates for doc generation
The container is versioned and pinned in the workflow YAML. Updates to the agent logic require a new container build and a version bump in all consuming workflows. There is no central control plane. Each product repo runs its own instance of the workflow.
Code Example: PR Analysis Step
Here is a simplified version of the PR analysis logic that decides whether a merged PR needs documentation:
import openai
from github import Github
def analyze_pr_for_docs(repo_name, pr_number, gh_token):
g = Github(gh_token)
repo = g.get_repo(repo_name)
pr = repo.get_pull(pr_number)
# Gather context
files_changed = [f.filename for f in pr.get_files()]
commit_messages = [c.commit.message for c in pr.get_commits()]
pr_body = pr.body or ""
prompt = f"""
Analyze this merged PR and determine if it requires documentation updates.
Files changed: {files_changed}
Commit messages: {commit_messages}
PR description: {pr_body}
Return JSON with:
- requires_docs: boolean
- reason: string
- affected_docs: list of doc file paths that likely need updates
"""
response = openai.ChatCompletion.create(
model="gpt-4",
messages=[{"role": "user", "content": prompt}],
response_format={"type": "json_object"}
)
return response.choices[0].message.contentThis function runs in a GitHub Actions step. The output JSON is stored as an artifact and consumed by the doc generation step.
Trade-Offs and Limitations
| Aspect | Benefit | Cost |
|---|---|---|
| Cross-repo coordination | Keeps product and docs in sync | Requires mapping file maintenance |
| SME review loop | Prevents incorrect docs from merging | Adds latency to doc updates |
| Stateless orchestration | Simple to reason about and debug | Context must be serialized between steps |
| Per-repo workflow instances | No single point of failure | Version drift across product repos |
| Nightly index rebuild | Keeps RAG context fresh | Up to 24-hour lag in doc corpus |
The system does not handle documentation that spans multiple product changes. If three PRs merge in quick succession and all affect the same doc section, the agent creates three separate doc PRs. The SME must manually reconcile them.
The agent also does not understand visual assets. If a product change requires a new screenshot or diagram, the agent generates a placeholder comment in the doc PR asking the SME to add the asset.
Technical Verdict
Use this pattern when:
- You have a clear boundary between product and documentation repositories
- SMEs are willing to review agent-generated PRs within a defined SLA
- You can maintain a mapping file linking code areas to documentation owners
- Your documentation follows consistent structure and style guidelines
Avoid this pattern when:
- Documentation and code live in the same repository (simpler tooling works)
- Your docs require heavy visual content or interactive examples
- SME availability is unpredictable or review latency is unacceptable
- You need real-time doc updates (the nightly index rebuild introduces lag)
The Aspire team’s implementation is production-grade but not turnkey. Replicating it requires investment in mapping file maintenance, SME training, and observability integration. The payoff is measurable: GitHub reports a 40% reduction in documentation lag time and a 60% decrease in stale doc issues filed by users.