Anthropic just released a reference implementation for autonomous vulnerability discovery. It’s not a product. It’s a blueprint showing how they think about agent security workflows after partnering with security teams through their Claude Mythos Preview program.
The repo (7,290 stars, trending #16 in Python) includes interactive Claude Code skills and an autonomous harness that runs a five-stage pipeline: recon, find, verify, report, patch. The harness is configured for C/C++ memory vulnerabilities using Docker and ASAN, but the architecture generalizes to other languages and vulnerability classes.
This is rare. Frontier labs rarely publish reference implementations with this much plumbing detail. The code reveals how Anthropic thinks about verification gates, sandboxing boundaries, and the gap between agent confidence and actual exploitability.
Pipeline Architecture
The autonomous harness runs five stages in sequence:
- Recon: Map the codebase. Identify attack surfaces, trust boundaries, and high-risk modules.
- Find: Generate candidate vulnerabilities using static analysis and pattern matching.
- Verify: Run dynamic checks (ASAN, fuzzing, or custom detectors) to confirm exploitability.
- Report: Deduplicate findings, assign severity, and generate structured output.
- Patch: Propose fixes, validate them in the same sandbox, and emit diffs.
Each stage is a separate agent invocation with its own prompt, tools, and success criteria. The harness passes state forward using JSON artifacts. If verification fails, the finding is discarded before it reaches the report stage.
This multi-stage design is the key architectural choice. It trades latency for precision. A single-pass agent would hallucinate more false positives. The verification gate forces the agent to prove exploitability before committing to a finding.
Verification Gates and Confidence Calibration
The verify stage runs ASAN (Address Sanitizer) inside a Docker container. The agent writes a test harness, compiles the target with ASAN enabled, and checks for crashes or sanitizer violations.
This is not symbolic execution or formal verification. It’s dynamic testing. The agent generates concrete inputs that trigger the suspected vulnerability. If ASAN doesn’t fire, the finding is dropped.
The README warns that this won’t work on every codebase out of the box. ASAN requires recompilation with specific flags. Some codebases have build systems that don’t cooperate. Some vulnerabilities (logic bugs, race conditions) don’t trigger sanitizers at all.
The verification gate reduces false positives but introduces false negatives. The agent might fail to generate the right test input. The build might fail. The vulnerability might be real but undetectable by ASAN.
This is the confidence calibration problem. The agent’s internal confidence score (from the LLM’s logits) doesn’t map cleanly to exploitability. The verification stage is a hard gate that forces the agent to produce evidence, not just a high confidence score.
Sandboxing and Isolation Boundaries
The harness runs all verification and patching steps inside Docker containers. Each container is ephemeral. The agent can’t persist state across invocations except through the JSON artifacts passed between stages.
The Dockerfile installs build tools, ASAN, and language-specific dependencies. The agent mounts the target codebase as a read-only volume during verification and read-write during patching.
This sandboxing strategy has three goals:
- Isolation: Prevent the agent from modifying the host filesystem or accessing network resources.
- Reproducibility: Each verification run starts from a clean state.
- Portability: The harness can run on any machine with Docker installed.
The isolation boundary is the Docker socket. The agent can execute arbitrary commands inside the container but can’t escape to the host. This is weaker than VM-level isolation but stronger than process-level sandboxing.
The README doesn’t specify resource limits (CPU, memory, disk). In production, you’d want to enforce quotas to prevent runaway builds or infinite loops during verification.
Interactive Skills vs. Autonomous Harness
The repo includes two execution modes: interactive skills and the autonomous harness.
The interactive skills (/threat-model, /vuln-scan, /triage, /patch, /customize) run inside Claude Code. They’re designed for human-in-the-loop workflows. A security engineer runs /threat-model to scope the attack surface, then runs /vuln-scan on specific modules. The agent proposes findings, the engineer triages them, and the agent generates patches on demand.
The autonomous harness runs the same logical flow without human intervention. It’s a batch job. You point it at a codebase, and it runs all five stages in sequence.
The separation teaches a lesson about agent design. Interactive agents need different affordances than batch agents. Interactive agents need undo, explain, and refine commands. Batch agents need checkpointing, retry logic, and structured logging.
The harness doesn’t include a web UI or dashboard. It emits JSON reports to stdout. You’d need to build your own frontend if you want to visualize findings or track remediation status over time.
Customization Surface
The /customize skill is the most interesting part of the interactive mode. It walks you through porting the harness to a new language, detector, or vulnerability class.
The customization surface includes:
- Language: Swap C/C++ for Rust, Go, or JavaScript.
- Detector: Replace ASAN with Valgrind, ThreadSanitizer, or a custom static analyzer.
- Vulnerability class: Retarget from memory bugs to SQL injection, XSS, or deserialization flaws.
- Build system: Adapt the Dockerfile to handle CMake, Bazel, or npm.
The skill generates a new Dockerfile, updates the verification prompts, and modifies the tool definitions. It doesn’t write the detector integration for you. You still need to know how to invoke your chosen tool and parse its output.
This is where the “reference, not a product” disclaimer bites. The harness assumes you have deep knowledge of your target language and toolchain. If your build system is exotic or your detector requires manual tuning, you’ll spend more time customizing than scanning.
State Management and Checkpointing
The harness doesn’t checkpoint intermediate state. If the verify stage crashes, you lose all the work from the find stage. You have to rerun the entire pipeline.
This is a deliberate simplification. Checkpointing adds complexity. You need to serialize agent state, handle partial failures, and decide when to resume vs. restart.
For a reference implementation, the tradeoff makes sense. For production use, you’d want checkpointing. Security scans can take hours on large codebases. Losing progress to a transient Docker failure is unacceptable.
The JSON artifacts passed between stages are the natural checkpoint format. You could write each artifact to disk and add a resume flag that skips completed stages.
Observability Gaps
The harness logs to stdout. Each stage prints a status message and the final JSON artifact. There’s no structured logging, no trace IDs, no metrics.
For production observability, you’d need:
- Trace IDs: Link all stages of a single scan together.
- Metrics: Track stage duration, verification success rate, and patch acceptance rate.
- Alerts: Fire when verification fails repeatedly or when the agent generates malformed patches.
- Audit logs: Record every tool invocation and every file modification.
The companion cookbook (SDK-only walkthrough) shows how to add basic logging using the Anthropic SDK’s message callbacks. You’d still need to wire it into your observability stack (Datadog, Honeycomb, or OpenTelemetry).
Failure Modes
The README lists several known failure modes:
- Build failures: The Dockerfile might not install the right dependencies. The build system might require manual configuration.
- ASAN false negatives: Some vulnerabilities don’t trigger sanitizers. Logic bugs and race conditions are invisible to ASAN.
- Patch validation failures: The agent might generate a patch that fixes the vulnerability but breaks the build or introduces new bugs.
- Timeout: Long-running verification steps might exceed the agent’s context window or Docker’s default timeout.
The harness doesn’t handle these gracefully. It crashes or emits an incomplete report. In production, you’d need retry logic, fallback strategies, and human escalation paths.
Comparison: Interactive vs. Autonomous Workflows
| Dimension | Interactive Skills | Autonomous Harness |
|---|---|---|
| Execution | Human-in-the-loop, step-by-step | Batch, end-to-end |
| Latency | Seconds per step | Minutes to hours |
| Precision | High (human triage) | Medium (verification gate) |
| Scalability | One repo at a time | Many repos in parallel |
| Observability | Claude Code UI | JSON logs to stdout |
| Customization | /customize skill | Manual Dockerfile edits |
| Checkpointing | Implicit (human memory) | None (restart on failure) |
Code Example: Verification Stage
The verification stage is the most complex. Here’s a simplified version showing how the agent invokes ASAN inside Docker:
import subprocess
import json
def verify_vulnerability(finding: dict, codebase_path: str) -> bool:
"""
Run ASAN verification for a candidate vulnerability.
Returns True if ASAN detects a violation, False otherwise.
"""
# Generate test harness from finding metadata
test_code = generate_test_harness(finding)
# Write test harness to temp file
with open("/tmp/test_harness.c", "w") as f:
f.write(test_code)
# Build with ASAN enabled
build_cmd = [
"docker", "run", "--rm",
"-v", f"{codebase_path}:/code:ro",
"-v", "/tmp/test_harness.c:/test.c:ro",
"anthropic/asan-builder",
"gcc", "-fsanitize=address", "-g",
"/code/target.c", "/test.c", "-o", "/tmp/test"
]
try:
subprocess.run(build_cmd, check=True, timeout=300)
except subprocess.CalledProcessError:
return False # Build failed, discard finding
# Run test and check for ASAN violations
run_cmd = [
"docker", "run", "--rm",
"-v", "/tmp/test:/test:ro",
"anthropic/asan-runner",
"/test"
]
result = subprocess.run(
run_cmd,
capture_output=True,
timeout=60
)
# ASAN violations appear in stderr
return "AddressSanitizer" in result.stderr.decode()This is pseudocode. The real harness uses the Anthropic SDK to let the agent generate the test harness dynamically. The agent also parses ASAN output to extract crash details and severity.
Technical Verdict
Use this harness when:
- You’re building a security agent pipeline from scratch and need a reference architecture.
- You have deep knowledge of your target language and toolchain.
- You’re willing to invest in customization (Dockerfile, prompts, tool definitions).
- You need a starting point for multi-stage verification workflows.
Avoid this harness when:
- You need a turnkey solution. This is a reference, not a product.
- Your codebase has an exotic build system or requires manual configuration.
- You need production-grade observability, checkpointing, or error handling out of the box.
- You’re scanning for vulnerability classes that don’t trigger dynamic detectors (logic bugs, race conditions).
The real value is the architecture. The five-stage pipeline, the verification gate, and the separation between interactive and autonomous modes are reusable patterns. The Docker sandboxing and JSON artifact passing are solid foundations for production systems.
The gap between reference and production is wide. You’ll need to add checkpointing, structured logging, resource limits, and retry logic. You’ll need to tune the prompts for your specific vulnerability classes. You’ll need to integrate with your CI/CD pipeline and your security team’s workflow.
But if you’re building a security agent from scratch, this is the best open-source reference implementation available. It shows how a frontier lab thinks about agent security workflows in practice.