mech.app

The mech.app newsletter

Agentic AI, minus the noise.

Get practical field notes on AI agents, automation, developer tools and security delivered to your inbox.

No spam. Unsubscribe anytime.

Dev Tools

ThinkRail's Agent Skill System: How JetBrains Built Self-Extending IDE Capabilities Around Living Specs

How ThinkRail lets agents create their own tools using living specs as durable knowledge and extension points as capability boundaries.

Source: github.com
ThinkRail's Agent Skill System: How JetBrains Built Self-Extending IDE Capabilities Around Living Specs

JetBrains shipped ThinkRail as an IDE where agents can write their own capabilities instead of waiting for plugin updates. The project hit 5,016 stars and #1 on GitHub Trending (TypeScript) by combining isolated workspaces, living specs as durable project knowledge, and a skill system that lets the pi coding agent extend itself based on how you actually work.

This is not a wrapper around an LLM chat interface. ThinkRail is a full IDE built around agent-extensibility as a core feature. The agent can create tools, register them as reusable skills, and invoke them across workspace boundaries without breaking existing state.

Architecture: Workspaces, Specs, and Skills

ThinkRail runs on three layers:

  1. Isolated workspaces using git worktrees. Each agent conversation gets its own filesystem view. Changes stay contained until you explicitly merge them.
  2. Living specs as structured project knowledge. These are not static docs. They evolve with the codebase and serve as the substrate for skill generation.
  3. Agent skills as registered capabilities. The agent can write a new skill, test it in isolation, and add it to the registry for future invocations.

The pi coding agent sits at the center. It decides when to invoke an existing skill versus writing a new one based on the task context and available specs.

Workspace Isolation Boundaries

Each workspace is a separate git worktree. The agent can:

  • Read the main branch state
  • Write changes to its own worktree
  • Invoke skills that operate on its isolated view
  • Propose merges back to main

This prevents a newly-created skill from corrupting the main branch or interfering with other active workspaces. If a skill fails, the worktree is disposable. You lose the experiment, not the project.

Living Specs as Durable Knowledge

Traditional documentation goes stale. Living specs are different:

  • They are versioned with the code
  • They include executable examples
  • They reference actual file paths and function signatures
  • The agent updates them when it changes the implementation

When the agent needs to create a new skill, it queries the spec registry for relevant context. The spec provides:

  • Expected input/output shapes
  • Existing patterns in the codebase
  • Constraints (security boundaries, API limits, state dependencies)

This makes specs a durable knowledge substrate instead of a brittle reference. The agent does not hallucinate tool signatures because the spec includes the actual contract.

Skill Registry: Invocation vs. Creation

The skill registry is a lookup table mapping task patterns to executable capabilities. When the agent receives a request, it:

  1. Parses the intent
  2. Queries the registry for matching skills
  3. If a match exists, invokes it with the current workspace context
  4. If no match exists, checks living specs for relevant patterns
  5. Generates a new skill, tests it in isolation, and registers it

Decision Logic

ConditionActionRationale
Exact skill matchInvoke existing skillReuse proven capability
Partial match + simple deltaCompose existing skillsAvoid redundant code
No match + spec existsGenerate new skill from specLeverage durable knowledge
No match + no specPrompt user for clarificationPrevent hallucinated tools
Skill fails in testRollback and logProtect workspace state

The registry does not blindly invoke. It checks workspace compatibility first. A skill written for a Node.js project will not run in a Python workspace, even if the task description matches.

Skill Generation Flow

Here is how the agent creates a new skill:

// Simplified skill generation flow
async function generateSkill(intent: string, workspace: Workspace): Promise<Skill> {
  // 1. Query living specs for relevant context
  const specs = await specRegistry.query(intent);
  
  // 2. Generate skill code using spec patterns
  const skillCode = await agent.generate({
    intent,
    specs,
    workspace: workspace.context,
    constraints: workspace.securityBoundaries
  });
  
  // 3. Test in isolated worktree
  const testResult = await workspace.test(skillCode);
  
  if (!testResult.success) {
    throw new SkillGenerationError(testResult.errors);
  }
  
  // 4. Register skill with metadata
  const skill = {
    id: generateSkillId(intent),
    code: skillCode,
    metadata: {
      createdAt: Date.now(),
      workspace: workspace.id,
      specs: specs.map(s => s.id),
      testCoverage: testResult.coverage
    }
  };
  
  await skillRegistry.register(skill);
  return skill;
}

The key is step 3: testing in isolation. The agent does not register a skill until it passes tests in the worktree. This prevents broken tools from polluting the registry.

Security Boundaries

Agent-generated skills are powerful and risky. ThinkRail enforces three boundaries:

  1. Filesystem isolation: Skills can only read/write within their workspace worktree
  2. Network sandboxing: Skills cannot make outbound requests unless explicitly allowed in the spec
  3. Registry permissions: Only skills that pass tests and match spec constraints get registered

The agent cannot escalate privileges. If a skill needs network access, the user must update the spec to allow it. The agent can propose the change, but it cannot self-approve.

Failure Modes

FailureImpactMitigation
Skill generates invalid codeTest fails, skill not registeredWorktree rollback
Skill passes tests but breaks in productionWorkspace corruptionGit reset to pre-skill state
Skill conflicts with existing skillRegistry collisionNamespace isolation by workspace ID
Spec drift (code changes, spec does not)Agent generates outdated skillSpec validation on commit

The most dangerous failure is spec drift. If the spec says a function takes two arguments but the actual implementation takes three, the agent will generate a broken skill. ThinkRail mitigates this with pre-commit hooks that validate specs against the codebase.

Extension Points

ThinkRail is adding full extension support next. This means:

  • Third-party skills can register in the global registry
  • Skills can depend on other skills (composability)
  • Users can override agent-generated skills with custom implementations

The extension API will expose:

  • Workspace lifecycle hooks (onCreate, onMerge, onDiscard)
  • Spec schema validation
  • Skill invocation middleware (logging, rate limiting, cost tracking)

This turns ThinkRail into a platform, not just an IDE. You can build domain-specific skill libraries (e.g., “Kubernetes deployment skills” or “React component generation skills”) and share them across teams.

Observability

The agent logs every skill invocation with:

  • Input parameters
  • Workspace state snapshot
  • Execution time
  • Output diff
  • Test results

This creates an audit trail. If a skill breaks something, you can replay the invocation in a fresh worktree and debug it without affecting the main branch.

The logs also feed back into the skill registry. If a skill consistently fails for certain input patterns, the agent can flag it for review or automatically deprecate it.

Deployment Shape

ThinkRail runs as a desktop Electron app. The agent backend is a Node.js process that:

  • Manages git worktrees
  • Runs the pi coding agent
  • Maintains the skill registry and spec index
  • Handles skill execution in isolated sandboxes

The frontend is a React app that renders:

  • The workspace tree (active worktrees and their status)
  • The conversation view (agent chat with tool step visualization)
  • The diff viewer (side-by-side comparison of workspace changes)
  • The skill inspector (registry browser with execution history)

All state lives locally. There is no cloud dependency for core functionality. You can use your own LLM API keys (OpenAI, Anthropic, local models via Ollama).

Technical Verdict

Use ThinkRail when:

  • You want an IDE that learns your project patterns and codifies them as reusable skills
  • You need isolated workspaces for agent experiments without risking the main branch
  • You value durable, versioned specs over static documentation
  • You are comfortable with agent-generated tools running in sandboxed environments

Avoid ThinkRail when:

  • You need a traditional IDE with stable, vendor-supported plugins
  • Your team cannot tolerate agent-generated code in the toolchain
  • You require air-gapped environments (the agent needs LLM API access)
  • You work in languages or frameworks without strong spec conventions (the skill system depends on structured knowledge)

The skill registry is the breakthrough. Most agentic IDEs treat the agent as a code generator. ThinkRail treats it as a capability builder. The agent does not just write code for you. It writes tools for itself, tests them, and adds them to its own toolkit. That is a different kind of automation.