Documentation Debt in Coding Agents: Why AGENTS.md Files Go Stale and How to Detect It

Coding agents like Claude Code and Cursor read repository instruction files (AGENTS.md, CLAUDE.md, .cursorrules) at the start of every session. These files tell the agent where tests live, how to run builds, and what conventions to follow. When the code changes but the instruction file does not, you get documentation debt. The agent follows outdated guidance, wastes tool calls on nonexistent paths, and sometimes fails tasks it should complete.

A new ArXiv paper (2611.09802v1) measures this cost directly. Researchers ran 732 agent trajectories across three real-world stale instructions and counted the extra tool calls, failed operations, and recovery patterns. The findings expose a hidden maintenance surface that most teams do not monitor.

The Problem: Instruction Files Are Read-Only Context with No Feedback Loop

Instruction files are loaded into the agent’s context window but never updated by the agent itself. When a test directory moves or a build script changes name, the file still points to the old location. The agent reads the stale instruction, attempts the operation, hits an error, and either recovers or fails.

This is different from code comments going stale. A stale comment might confuse a human reviewer once. A stale instruction file confuses the agent in every session where that instruction is relevant.

Key staleness patterns identified:

Test command drift: The instruction says npm test, but the repo switched to pnpm test.
Path obsolescence: The file points to src/components/Button.tsx, but the file moved to src/ui/Button.tsx.
Procedural rot: The instruction describes a three-step process for adding a feature, but step two no longer applies after a refactor.

The paper studied three real-world cases from active open-source repositories. Each represented a different kind of guidance. Agents ran 22 maintenance tasks under three versions of each instruction: left stale, corrected, and removed entirely.

Measured Cost: Extra Tool Calls and Obsolete Path Operations

On a stale test command, Claude Sonnet 3.5 needed about two extra tool calls per run compared to the corrected version. That is a 33% increase in tool invocations for that task. The agent would try the old command, see it fail, search for the correct command, and retry.

For path-based instructions, the cost was harder to isolate. Agents sometimes followed the stale path even when correct information existed elsewhere in the repository. In other cases, they ignored the stale instruction and found the correct path through file system exploration. The variance depended on task relevance: if the stale instruction was central to the task, the agent followed it. If it was peripheral, the agent often skipped it.

What agents did not do:

None of the agents repaired the underlying instruction file, even after recovering from the immediate failure.
Agents treated the instruction file as immutable context, not as a document to maintain.

Removing the stale instruction entirely prevented most obsolete path operations. Correcting it eliminated them completely.

Detection Mechanisms: Static Analysis and Diff-Based Triggers

The paper proposes two detection strategies:

1. Static path validation

Parse the instruction file for file paths, directory references, and shell commands. Check each reference against the current repository structure. Flag mismatches as potential staleness.

import os
import re

def validate_instruction_paths(instruction_file, repo_root):
    with open(instruction_file) as f:
        content = f.read()
    
    # Extract file paths (simple regex, extend for robustness)
    paths = re.findall(r'`([^`]+\.(ts|py|md|json))`', content)
    
    stale = []
    for path, _ in paths:
        full_path = os.path.join(repo_root, path)
        if not os.path.exists(full_path):
            stale.append(path)
    
    return stale

This catches moved files and renamed directories but misses semantic drift (a command that still exists but behaves differently).

2. Diff-based triggers

Monitor commits that touch files or directories mentioned in the instruction file. When a referenced path changes, flag the instruction file for review.

# .github/workflows/instruction-check.yml
name: Instruction File Staleness Check
on:
  push:
    paths:
      - 'src/**'
      - 'tests/**'
      - 'package.json'

jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Extract paths from AGENTS.md
        run: |
          grep -oP '`\K[^`]+(?=`)' AGENTS.md > referenced_paths.txt
      - name: Check if changed files match references
        run: |
          git diff --name-only HEAD~1 HEAD | grep -Ff referenced_paths.txt && \
          echo "::warning::AGENTS.md may need update" || true

This approach is noisy (not every change to a referenced file invalidates the instruction) but catches the majority of drift cases.

Architecture: Where Instruction Files Fit in Agent Orchestration

Instruction files sit at the orchestration layer, not the tool layer. They are loaded once per session and influence every subsequent tool call. This makes them a global state dependency with no versioning or rollback mechanism.

Typical agent session flow:

Session init: Load instruction file into system prompt or context window.
Task planning: Agent reads instructions to understand repository conventions.
Tool execution: Agent calls file read, shell exec, or code edit tools.
Error recovery: If a tool call fails, agent retries or searches for alternatives.
Session end: Instruction file is discarded, not updated.

The instruction file is read-only from the agent’s perspective. It cannot write back corrections, flag inconsistencies, or version its own understanding.

Component	Responsibility	Staleness Risk
Instruction file	Provide repository-level guidance	High (no automated sync with code changes)
System prompt	Define agent behavior and constraints	Low (versioned with agent deployment)
Tool call logs	Record agent actions	None (append-only)
Repository state	Source of truth for current structure	None (always current)
CI/CD pipeline	Enforce quality gates	Medium (if instruction checks not included)

When to Gate on Instruction File Staleness

Not every team needs instruction file validation in CI. The cost/benefit depends on agent usage frequency and instruction file complexity.

Add staleness checks if:

Agents run on every pull request or commit.
Instruction files contain more than ten specific paths or commands.
Agents frequently fail tasks due to missing files or incorrect paths.
Multiple developers contribute to the repository (higher chance of uncoordinated changes).

Skip staleness checks if:

Agents are used sporadically (manual invocation only).
Instruction files are minimal (generic guidance, no hardcoded paths).
The repository structure is stable (no frequent refactors).

A middle-ground approach: run staleness checks on a schedule (weekly) rather than per-commit. Flag issues as warnings, not blockers.

Failure Modes and Recovery Patterns

The paper identified three recovery patterns when agents encountered stale instructions:

1. Immediate retry with search

Agent tries the stale command, sees it fail, searches the repository for the correct command, and retries. This adds 1-3 tool calls per failure.

2. Fallback to exploration

Agent ignores the stale instruction and explores the file system to find the correct path. This works when the task does not strictly depend on the instruction.

3. Task abandonment

Agent follows the stale instruction, fails, and cannot recover. This happened in 8% of stale instruction runs in the study.

None of the agents flagged the instruction file itself as the root cause. They treated failures as task-specific issues, not documentation issues.

Technical Verdict

Use instruction file staleness detection when:

Agents are integrated into CI/CD and run frequently.
Instruction files reference specific paths, commands, or multi-step procedures.
You want to reduce wasted tool calls and improve agent task success rates.

Avoid or defer when:

Agent usage is infrequent or experimental.
Instruction files are short and generic.
You lack the tooling to automate path validation or diff-based triggers.

The cost of stale instruction files is real but not catastrophic. A 33% increase in tool calls for affected tasks is measurable, but agents often recover. The bigger risk is silent failures: tasks that succeed despite following stale guidance, leaving incorrect assumptions embedded in the agent’s workflow.

If you are running agents in production, treat instruction files as code. Version them, review them on refactors, and validate them in CI. If you are experimenting, accept the occasional extra tool call as the cost of flexibility.

Agentic AI, minus the noise.