> ## Documentation Index > Fetch the complete documentation index at: https://vietbui.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Context Compression Strategies > When agent sessions grow long, compression is mandatory. The wrong optimization target is tokens per request (minimize context size). The correct target is t… # Context Compression Strategies When agent sessions grow long, compression is mandatory. The wrong optimization target is *tokens per request* (minimize context size). The correct target is **tokens per task** — total tokens to complete a task, including re-fetch costs when compression loses critical information. ## Five Strategies ### 1. Anchored Iterative Summarization (recommended) Maintain a structured persistent summary with named sections: session intent, files modified, decisions made, next steps. When compaction triggers, summarize only the newly-truncated span and **merge** it into the existing summary — don't regenerate. **Why it works**: Structure forces preservation — each section is a dedicated slot that must be filled. Incremental merging prevents compounding loss across multiple compression cycles. **When to use**: Long-running coding sessions, multi-step tasks, anything where the agent needs to track file modifications and decisions across context windows. ``` [SUMMARY] Intent: Implement auth middleware for the API Files modified: src/auth/middleware.ts (added), src/routes/api.ts (updated) Key decisions: Using JWT with 1h expiry; refresh token in httpOnly cookie Blocked on: CORS configuration for cross-origin refresh token flow Next: Write tests for token refresh path ``` *** ### 2. Opaque Compression Produce a compressed representation optimized for reconstruction fidelity rather than human readability. Achieves highest compression ratios (99%+) but sacrifices interpretability. **Why it works**: Optimizes for what the model needs to reconstruct context, not what a human would find readable. **When to use**: High-volume automated pipelines where you don't need to inspect the compressed summary; when compression ratio matters more than debuggability. **Caveat**: Cannot verify what was preserved. Harder to debug when compression loses critical information. *** ### 3. Regenerative Full Summary Generate a fresh detailed summary from scratch on each compression trigger. **Why it's worse**: Each regeneration can lose details that prior compressions handled correctly. Compounding across multiple cycles causes drift. Produces readable output but less reliable than anchored iterative across long sessions. **When acceptable**: Short sessions with a single compression event; when simplicity matters more than accuracy. ### 4. CCR — Reversible Compression with Retrieval Compress aggressively into the context window; store originals locally; expose a retrieval tool so the LLM fetches full content on demand. The model decides what to retrieve at inference time rather than the compressor predicting relevance upfront. **When CCR wins**: large tool outputs where the relevant 10% varies by query — the agent can't predict upfront what it needs. CCR avoids the prediction problem by deferring selection to inference. **Limitation**: requires the LLM to recognize when it's missing information and call the retrieval tool. If the model doesn't know what it dropped, it won't ask. Anchored summarization is more reliable when critical information is structurally predictable. **Implementation**: [Headroom](/entities/headroom) ships CCR via `headroom_retrieve` MCP tool. *** ### 5. Adaptive Guideline-Optimized Compression (research: Acon) Use an LLM to compress context guided by a **learned prompt** — the compression guideline is optimized from task failure signals, not handcrafted. Applies separately to interaction history (when length > threshold) and raw observations (when observation > threshold). **How it works**: Run agent with and without compression; find tasks where compression causes failure; feed contrastive pairs to an optimizer LLM to refine the guideline. Two alternating phases: utility maximization (fix failures) and compression maximization (shorten successes). Gradient-free — works with any API model. **Why it can outperform fixed-prompt compression**: The guideline learns environment-specific signals — what types of information cause failure when dropped (e.g., API auth tokens, account balances, due dates). Generic prompts miss these domain constraints. **When to use**: Multi-step agentic tasks with heterogeneous tool outputs; when generic summarization degrades task performance; when you have training trajectories to optimize from. **Cost warning**: History compression breaks KV-cache prefix stability, forcing recomputation — this can make total API cost *higher* than no compression. Observation compression avoids this: compress before the observation enters history. Prefer observation compression in cost-sensitive settings. **Distillation**: Once the large-model guideline is optimized, distill the compressor into a small model (e.g., Qwen3-14B via LoRA) to eliminate the overhead of calling a large LLM per step. Distilled compressors retain >95% of teacher accuracy across all benchmarks. **Benchmark results** (AppWorld / OfficeBench / 8-obj QA): 26–54% peak token reduction; maintained or improved task performance. Small agent improvement (Qwen3-14B): +32% AppWorld, +20% OfficeBench, +46% QA. Source: Acon paper (KAIST + Microsoft, 2025; AppWorld/OfficeBench/8-obj QA benchmarks) *** ## Content-Type-Aware Routing Generic text compression treats all context as prose. A more effective approach dispatches by content type: | Content Type | Compressor | Why | | --------------------------- | -------------------------- | ---------------------------------------------------------------------------------- | | JSON / structured data | SmartCrusher (rule-based) | Arrays of dicts compress far more than prose — deduplicate keys, elide null fields | | Source code | CodeCompressor (AST-aware) | AST structure enables semantic pruning — remove docstrings, redundant signatures | | Prose / logs / unstructured | Kompress-base (ML model) | Structure-agnostic; trained on agentic traces | **Why it matters**: applying a prose compressor to JSON wastes compression potential; applying an ML model to source code misses AST-level redundancy. Content-type routing makes compression composition additive. **Implementation**: [Headroom](/entities/headroom) ContentRouter automates this dispatch. *** ## Key Results from Compression Evaluation (Factory.ai, 2025) Factory.ai evaluated three production compression approaches on 36,611 messages from real software engineering agent sessions. Grading via GPT-5.2 LLM judge, six dimensions (accuracy, context awareness, artifact trail, completeness, continuity, instruction following). | Method | Overall | Accuracy | Context | Artifact | Continuity | | ----------------------------------- | -------- | -------- | -------- | -------- | ---------- | | Factory (anchored iterative) | **3.70** | **4.04** | **4.01** | **2.45** | 3.80 | | Anthropic (regenerative structured) | 3.44 | 3.74 | 3.56 | 2.33 | 3.67 | | OpenAI (opaque /compact) | 3.35 | 3.43 | 3.64 | 2.19 | **3.77** | Compression ratios: Factory 98.6%, Anthropic 98.7%, OpenAI 99.3%. Key finding: **artifact trail is unsolved** — all methods scored 2.19–2.45 / 5.0 on tracking file modifications. Summarization alone cannot reliably track file state across long sessions; dedicated artifact indexing in the harness is likely needed. Accuracy gap: Factory 4.04 vs OpenAI 3.43 (0.61 difference) — reflects file paths and error codes surviving compression. OpenAI's opaque endpoint treats file paths as "low-entropy content" and discards them, which is catastrophically wrong for coding agents. *** ## Token Budget Allocation Understanding where tokens go helps decide what to protect vs. compress: | Component | Typical Range | Compress? | | ----------------------- | ---------------- | ------------------------------------ | | System prompt | 500–2000 tokens | Never | | Tool definitions | 100–500 per tool | No | | Active task state | Variable | Never | | Critical decisions | Variable | No — move to summary | | Recent turns (last 3–5) | Variable | No | | Tool outputs (current) | Variable | Partial — keep head+tail | | Old message history | Grows unbounded | Yes — primary compression target | | Retrieved documents | Often largest | Selective — mask served-purpose docs | ## KV-Cache Optimization KV-cache hit rate determines recomputation cost. Cache hits require prefix stability — the beginning of the prompt must be identical across requests. **Design rules:** 1. Place system prompts and tool definitions at the very start — they're stable across turns 2. Never put timestamps, session IDs, or request-specific data in the system prompt 3. Keep dynamic content (current task, user query) at the end 4. Consistent formatting across requests — even whitespace changes invalidate cache ```python theme={null} # Cache-unfriendly: timestamp in system prompt system = f"Current time: {datetime.now().isoformat()}\nYou are..." # Cache-friendly: stable system prompt, dynamic data appended system = "You are a code assistant. Current time is provided per-request." ``` ## Practical Decision Rule * **Single session, no explicit complexity**: default to anchored iterative summarization * **High-volume pipeline, compression ratio critical**: opaque compression * **Quick prototype, one compression expected**: regenerative is fine * **Never**: aggressive deletion without a summary — you will lose file modification history ## Clear Over Compact — Now Community Consensus Matt Pocock argues against compacting and prefers hard context clears: > "I much prefer my AI to behave like the guy from Memento because this state is always the same. Every time you do it, you clear and you go back to the beginning." His argument: compaction introduces "sediment" — each compressed summary is imperfect, and multiple cycles compound drift. Clearing gives a deterministic, reproducible starting state. **Updated (2026-05-04)**: Among experienced practitioners running harness-based AFK coding workflows (r/ClaudeCode + r/opencodeCLI, 2026-05, n≈30), clear-over-compact is **majority practice** — not a contrarian position. Every active framework in that community (GSD, Dangeresque, SandCastle, vanilla loops) enforces fresh context per task. **This evidence is scoped**: the sample is \~30 experienced developers self-selected into harness-heavy workflows on two subreddits. It does not generalize to interactive/exploratory sessions, non-engineer users, or workflows without durable filesystem state. **When clear-over-compact is valid**: filesystem stores all state (issue files, commits, PRDs); harness re-injects state at session start. The key design principle: **make clearing safe, then prefer clearing**. **When compact wins over clear:** * No durable filesystem layer — clearing loses state that can't be reconstructed * Interactive debugging sessions where the chain of reasoning *is* the artifact * Exploratory sessions where you don't yet know what to externalize to files * Conversational research where history continuity is the product * Long sessions with a non-engineer who can't reconstruct context from files **When anchored iterative summarization is still better**: any of the "when compact wins" cases above. **OpenCode hook extension**: OpenCode exposes `experimental.session.compacting` — a hook that lets plugins inject domain state into compaction summaries or replace the compaction prompt entirely. This enables custom anchored iterative summarization at the harness level, more reliable than system-prompt instructions. See [Claude Code vs OpenCode Plugin Systems](/comparisons/claude-code-vs-opencode-plugins). ## Strategic Compaction Checkpoints From ECC's `strategic-compact` skill — explicit heuristics for *when* to compact: **Compact at logical seams:** * After research/exploration, before implementation begins * After completing a milestone, before starting the next task * After debugging session ends, before resuming feature work * After a failed approach, before trying a different strategy **Do not compact:** * Mid-implementation — you'll lose variable names, file paths, partial state * Mid-debugging — you'll lose the chain of reasoning that is the artifact **Token optimization settings** that complement compaction (from [Everything Claude Code (ECC)](/entities/everything-claude-code)): * `CLAUDE_AUTOCOMPACT_PCT_OVERRIDE=50` — compact at 50% of context (vs default 95%) — prevents quality degradation from near-full context * `MAX_THINKING_TOKENS=10000` — caps hidden thinking cost per request (claimed \~70% reduction) * `CLAUDE_CODE_SUBAGENT_MODEL=haiku` — subagents use a cheaper model tier All claimed impact numbers from ECC README; not independently verified. The pattern itself aligns with community consensus on token efficiency. *** ## Serialization Format (orthogonal layer) Compression (selection) and serialization (encoding) are independent axes: * Compress first: select what context to keep * Then serialize efficiently: encode it with minimal token overhead Schema-first formats (ONTO, TOON) achieve 40-60% reduction on structured data vs JSON. Most valuable for: large homogeneous datasets injected into prompts. See [LLM Serialization Formats](/concepts/llm-serialization-formats). *** ## Research: Learned Soft Compression (SAC) Distinct from the operational strategies above — SAC is a model architecture that produces compressed KV representations for serving. Not a drop-in tactic for Claude Code sessions. **Semantic-Anchor Compression (SAC)** — Northeastern University / NiuTrans Research, 2025. Code: [https://github.com/lx-Meteors/SAC](https://github.com/lx-Meteors/SAC) Core argument: autoencoding-based training objectives (ICAE, 500xCompressor, EPL) conflict with downstream task requirements — AE gradient and LM gradient are nearly orthogonal in parameter space, so optimizing one impairs the other. **SAC method**: select anchor tokens directly from context (uniform chunking at compression ratio `r`), augment each with a learnable anchor embedding, apply bidirectional attention in the encoder (anchors see full context). Decoder remains causally masked. No autoencoding objective — trains on downstream task loss only. **Results at 15× compression (vs EPL, second-best baseline), Llama-3.2-1B backbone:** * In-domain MRQA: +6.7% F1 / +8.2% EM * Out-of-domain MRQA: +6.9% F1 / +9.2% EM * vs ICAE: +23.5% F1 / +26.8% EM * Long-context summarization (32K input): SAC 18.49 avg ROUGE-1 F1 vs EPL 17.61 * Scales to 3B and 8B without diminishing returns Limitations: requires LoRA fine-tuning; benchmarks limited to MRQA/QMSum/GovReport; frontier-scale generalizability unverified. *** ## Related Pages * [Context Degradation Patterns](/concepts/context-degradation) — the failure modes compression prevents * [Agent Harness](/concepts/agent-harness) — where compaction fits in the harness component model * [Ralph Loop](/concepts/ralph-loop) — filesystem as durable state across clean context windows (the complement to compression) * [Dynamic Context Pruning (DCP)](/concepts/dynamic-context-pruning) — mid-session context reduction via Compress tool + deduplication + purge-errors; complements compaction * [Everything Claude Code (ECC)](/entities/everything-claude-code) — ECC's `strategic-compact` skill and token optimization settings