Skip to main content

​
Self-Improving Agent Setup

An agent environment (instructions, memory, skills, hooks, checks) that changes itself in response to real work so the same mistake or explanation is not needed twice. This applies to any coding agent (Claude Code, Codex, OpenCode, Copilot); the model weights stay fixed and only the harness changes. Distinct from Self-Healing Loop, which recovers from a failure within a run. Self-improvement changes the harness between runs.

​
Spectrum

​
What the evidence agrees on

  1. The bottleneck is compliance, not rule count. One nightly review found 146 of 153 lessons were already written down and simply not followed. Adding more prose rules raises context cost without raising adherence. See Rules vs. Hooks.
  2. Enforce what must always happen. Prose is advice under context pressure; a hook that blocks the action, or a check that goes red, is not. Code quality improved from checks going red, not from the agent “learning”.
  3. Gate what enters persistent memory. Practitioners: “only add a line I had to correct twice”. Harness²: only “evidence-supported lessons” enter the improvement harness. Both reject self-assessed lessons.
  4. Memory decays. Date every entry, cap the file size, re-check old lessons (one user saw ~6% of 114 lessons overruled later, some because the outside world changed). Once a lesson has been followed for weeks without a miss, move it from the always-loaded file to the procedure doc for that job.
  5. Keep always-loaded context small. Generated context files lowered success ~2–3% and raised cost > 20% in the ETH AGENTS.md study (Less Is More Claude Md, secondary report). Put local learnings next to the code (subdirectory instruction files load only when that folder is touched) and load the rest on demand.
  6. Prune. Remove polling monitors, periodic re-injection, and customisations that the platform has since shipped natively.

​
Open disagreement

Startup memory injection. One long-running multi-agent setup injects ~35k tokens of memory at session start and reports it works; others cite the ETH study and argue for progressive disclosure. The study measured repository context files on single-issue tasks, not per-agent working memory across long-running sessions, so it does not settle the question. Treat it as unresolved.

​
Minimal recipe (consensus, not validated)

  1. One small always-loaded file of non-obvious rules; everything else linked.
  2. A dated lessons file with a size cap; add only on second correction.
  3. A command that, after a fix, writes the rule where that workflow will see it next time, as a hook or check if it can be measured.
  4. A periodic prune: remove lessons that are now enforced, overruled, or redundant with native features.
  5. A human accepts every change to the harness.

​
How this wiki’s setup maps

mistakes/raw-log.md → capture-mistake → global-prevention-rules.md (≤ 45-line cap, dated entries) is levels 1 and 3. The judge-driven Preference Feedback Loop is level 3 with a cross-vendor evaluator. Promotion to hooks/guards is level 2. Levels 4 and 5 are not adopted.

​
Linked pages