Most AI coding agent demos fail in production because the model is too slow, too expensive, or hallucinates. That’s not what happened here. Across nine local projects using the same 45GB model on the same hardware, the autonomous pass rate climbed from 0% to 100% without touching the model. The bugs were in the orchestration layer, the state tracking, and a third axis the team didn’t anticipate.
This is field data from ForgeFlow’s Part 6 series. No cloud APIs during execution. Same local model throughout. The path from 29% to 100% pass rate exposes the plumbing failures that break coding agents before the LLM ever gets a chance to fail.
The Scoreboard
Nine projects, same setup, escalating complexity:
| Project | Pass Rate | CL Rules | Key Change |
|---|---|---|---|
| repo-jwt | 0% | 0 | No design rules existed |
| todo-api | 67% | ~10 | Context files added |
| bookmark-api | 100% | ~20 | Full information pipeline |
| expense-tracker | 70% | 32 | New failure patterns emerged |
| rating-api | 73% | 32 | DB fixture issues |
| library-api | scrapped | 35 | Architecture gap (multi-model FK) |
| event-api | 80% | 35 | Setup script pattern validated |
| habit-tracker | 44% | 39 | Route tasks collapsed |
| contact-book | 100% | 43 | All axes aligned |
The 100% figure on Project 9 is not an aggregate. It’s a controlled checkpoint: eight tasks, thirty-one tests, four minutes, zero manual intervention. After fixing the route-task failure pattern from Project 8, the same local model completed a comparable route-heavy project autonomously.
Project 6 was scrapped mid-execution. The orchestrator couldn’t state-track multi-model foreign-key setup scripts. Rather than pollute the loop data with a mismatched setup, the team halted and redesigned baseline infrastructure.
What CL Rules Actually Are
CL stands for Context Layer. These are not prompt templates. They’re structured design constraints fed into the agent’s context window before each task execution.
A CL rule looks like this:
rule_id: db_fixture_order
scope: setup_scripts
constraint: |
Database fixtures must initialize in dependency order:
1. User model
2. Foreign-key dependent models
3. Junction tables
validation: |
Check for FK constraint errors in test output.
If present, reorder fixture creation.By Project 9, the team had 43 CL rules. They covered:
- File structure conventions (where to put migrations, fixtures, routes)
- Execution order constraints (setup before tests, migrations before seeds)
- State validation checkpoints (database schema matches model definitions)
- Failure recovery patterns (retry with explicit FK ordering on constraint errors)
The rules grew organically. Each new failure mode generated a rule. The orchestrator injected relevant rules into the agent’s context based on task type.
The Two-Axis Model That Wasn’t Enough
Part 5 of ForgeFlow introduced the formula:
System Reliability ≈ DCR × Information Quality
DCR (Design, Code, Review) measures orchestration completeness. Information Quality measures how much relevant context the agent receives before acting.
This held through Project 3. Then Project 4 introduced database fixtures, and the pass rate dropped from 100% to 70%. Same model, same hardware, same DCR pipeline. The information was complete, but the agent kept generating fixtures in the wrong order, triggering foreign-key constraint errors.
The bug wasn’t in the model’s reasoning. It was in the orchestrator’s inability to enforce execution order across multi-file changes. The agent would write User and Post models correctly, then generate a Post fixture before the User fixture, causing the test suite to fail on FK violations.
The Third Axis: State Tracking Across Multi-Step Execution
The team added a state checkpoint layer between DCR stages. Before moving from Design to Code, the orchestrator now validates:
- Dependency graph completeness: Are all required models defined before dependent models?
- Execution order constraints: Do setup scripts respect FK dependencies?
- Rollback boundaries: If a test fails, which files need to revert?
This isn’t a new stage in the pipeline. It’s a validation gate that runs between existing stages. The orchestrator builds a dependency graph from the agent’s design output, checks it against CL rules, and either proceeds or injects a correction prompt.
Example state checkpoint:
def validate_fixture_order(design_output, cl_rules):
models = extract_models(design_output)
fixtures = extract_fixtures(design_output)
for fixture in fixtures:
dependencies = get_fk_dependencies(fixture, models)
for dep in dependencies:
if not fixture_exists_before(dep, fixtures):
return ValidationError(
f"Fixture {fixture} depends on {dep}, "
f"but {dep} is not initialized first"
)
return ValidationSuccess()After adding state checkpoints, Project 7 hit 80% pass rate. Project 8 dropped to 44% because of a new failure mode: route tasks collapsed. The agent would generate all CRUD routes in a single file instead of splitting them by resource, violating the file structure CL rules.
The fix was another checkpoint: validate file structure before code generation. If the design output specifies multiple resources, enforce separate route files.
Project 9 incorporated all checkpoints. 100% pass rate.
Failure Modes That Weren’t Model Failures
1. Fixture Initialization Order
Symptom: Tests fail with FK constraint errors.
Root cause: Agent generates fixtures in alphabetical order, not dependency order.
Fix: State checkpoint validates dependency graph before code generation. If violations exist, inject correction prompt with explicit ordering.
2. Route File Collapse
Symptom: All CRUD routes land in a single file, violating modularity rules.
Root cause: Agent optimizes for token efficiency, not file structure conventions.
Fix: CL rule enforces one resource per route file. State checkpoint validates file count matches resource count.
3. Migration Drift
Symptom: Database schema doesn’t match model definitions after migrations run.
Root cause: Agent writes migrations, but orchestrator doesn’t validate schema state before proceeding.
Fix: State checkpoint runs schema introspection after migrations. If drift detected, inject correction prompt with schema diff.
4. Setup Script Gaps
Symptom: Tests fail because database isn’t initialized.
Root cause: Agent assumes setup scripts exist, but orchestrator doesn’t enforce their creation.
Fix: CL rule requires setup script for any project with database dependencies. State checkpoint validates setup script exists before test execution.
The Infrastructure Stack
Local execution means no API rate limits, no token costs, and no network latency. It also means you own the entire failure surface.
Hardware: Single machine, 64GB RAM, RTX 4090 (24GB VRAM).
Model: 45GB local LLM (likely Llama 70B quantized, though not specified).
Orchestrator: Custom Python layer managing DCR pipeline, state checkpoints, and CL rule injection.
Execution environment: Docker containers for isolated test runs. Each project gets a fresh container to avoid state pollution.
State persistence: SQLite database tracking task history, checkpoint results, and CL rule applications. Allows rollback to any previous checkpoint.
Observability: Structured logs for each DCR stage, checkpoint validation, and agent prompt/response pairs. Logs feed into a simple dashboard showing pass/fail rates per project.
When Local Execution Makes Sense
Local coding agents work when:
- You have consistent hardware (no cloud cost variance).
- Your projects fit a repeatable pattern (CRUD APIs, not novel architectures).
- You can afford upfront orchestrator engineering (state checkpoints, CL rules).
- You need deterministic execution (same input, same output, no API jitter).
Local execution breaks when:
- Projects require architectures outside your CL rule coverage.
- You need multi-model collaboration (local orchestration gets complex fast).
- Your hardware can’t fit the model (45GB is a hard floor for code quality).
- You’re prototyping (cloud APIs are faster to iterate on).
Technical Verdict
The 100% pass rate on Project 9 is real, but narrow. It applies to a specific project type (route-heavy CRUD API) after 43 CL rules and multiple state checkpoint iterations. The model didn’t change. The orchestration layer absorbed all the complexity.
Use local coding agents when you have a repeatable project pattern, consistent hardware, and the engineering capacity to build state checkpoints. The upfront cost is high, but the marginal cost per project drops fast.
Avoid local coding agents when you’re exploring novel architectures, prototyping quickly, or lack the infrastructure to manage state tracking across multi-step execution. Cloud-hosted agents with simpler orchestration will ship faster.
The real lesson: most coding agent failures aren’t model failures. They’re orchestration failures, state tracking gaps, and missing execution order constraints. Fix the plumbing before you swap the model.