At a Glance
Context Engineering is the practice of designing the context passed to AI with "as little as possible, but as much as necessary."
Rather than making AI read everything, you narrow down the goal, constraints, relevant files, and verification method so AI can pick its next move without hesitation.
Good context is not about quantity — it’s about selection. Reduce what’s unnecessary; never omit what’s needed.
Context Rot
A larger context window does not make an LLM smarter. Cramming in too much information causes Lost in the Middle — important details get buried in the center — or Recency Bias — recent information gets overweighted — blunting the model’s judgment.
This is called context rot.
treemap-beta
"Context window"
"Smart zone": 60:::smart
"Dumb zone": 40:::dumb
classDef smart fill:#0891b2,stroke:#67e8f9,color:#ecfeff
classDef dumb fill:#7a1748,stroke:#ff2e88,color:#ffe8f4The goal of Context Engineering is not to fill the context window, but to keep the necessary information visible.
Input, Cached Input & Output Tokens
Every turn, what sits in the context window is billed as three kinds of tokens. Knowing them explains why long sessions cost what they do.
| Token type | What it is | Relative cost |
|---|---|---|
| 🟥 Input | New text sent this turn — prompt, files, tool results | Standard |
| 🟦 Cached input | Context resent from earlier turns, served from cache | Cheapest (big discount) |
| 🟩 Output | Text the model generates back to you | Most expensive |
Across a long chat, most of the window is cached input (the history is re-sent every turn), so caching is what keeps cost sane. You save most by trimming output and the input you add.
💡 The turn-by-turn slides that follow color each block by these token types.
Context Window: Start (turn 1)
Start is nearly ideal. Only the always-on System & tools are loaded — skill descriptions, copilot-instructions, MCP server tool schemas, default tool definitions — leaving plenty of working space. On the very first turn, those are sent fresh, so they count as input tokens.
Input tokens (sent this turn) · Free space / buffer
treemap-beta
"Context window"
"System & tools": 10:::used
"Free space": 75:::free
"Buffer": 15:::free
classDef used fill:#7a1748,stroke:#ff2e88,color:#ffe8f4
classDef free fill:#0891b2,stroke:#67e8f9,color:#ecfeffContext Window: Custom Agent (turn 1)
Switching to a custom agent adds that agent’s instruction to the context. Both system & tools and the custom agent are sent fresh — all input tokens on this turn.
Input tokens (sent this turn) · Free space / buffer
treemap-beta
"Context window"
"System & tools": 10:::used
"Custom Agent": 10:::used
"Free space": 65:::free
"Buffer": 15:::free
classDef used fill:#7a1748,stroke:#ff2e88,color:#ffe8f4
classDef free fill:#0891b2,stroke:#67e8f9,color:#ecfeffContext Window: Prompt (turn 1)
You write a prompt. For example, asking to “add a test” may cause a relevant skill and the path instruction for tests to be loaded. Everything in the window is fresh input tokens on this turn.
Input tokens (sent this turn) · Free space / buffer
treemap-beta
"Context window"
"System & tools": 10:::used
"Custom Agent": 10:::used
"Path instruction": 10:::used
"Skill": 10:::used
"Prompt": 10:::used
"Free space": 35:::free
"Buffer": 15:::free
classDef used fill:#7a1748,stroke:#ff2e88,color:#ffe8f4
classDef free fill:#0891b2,stroke:#67e8f9,color:#ecfeffContext Window: Sub-agent (turn 1)
The agent might call sub-agents — for example to explore the codebase, query a database, or run a review — and merge their summary back into the main context. Everything in the window is fresh input tokens on this turn.
Input tokens (sent this turn) · Free space / buffer
treemap-beta
"Context window"
"System & tools": 10:::used
"Custom Agent": 10:::used
"Path instruction": 10:::used
"Skill": 10:::used
"Prompt": 10:::used
"Sub-agent data": 10:::used
"Free space": 25:::free
"Buffer": 15:::free
classDef used fill:#7a1748,stroke:#ff2e88,color:#ffe8f4
classDef free fill:#0891b2,stroke:#67e8f9,color:#ecfeffContext Window: Memory (turn 1)
As you keep working in the repo, memories are generated and may be dynamically loaded when needed. Everything in the window is still fresh input tokens on this turn.
Input tokens (sent this turn) · Free space / buffer
treemap-beta
"Context window"
"System & tools": 10:::used
"Custom Agent": 10:::used
"Path instruction": 10:::used
"Skill": 10:::used
"Prompt": 10:::used
"Sub-agent data": 10:::used
"Memory": 10:::used
"Free space": 15:::free
"Buffer": 15:::free
classDef used fill:#7a1748,stroke:#ff2e88,color:#ffe8f4
classDef free fill:#0891b2,stroke:#67e8f9,color:#ecfeffContext Window: Output (turn 1)
The LLM responds. Its Answer is appended to the context — those are output tokens, charged at a different (typically higher) rate than input tokens.
Input tokens (sent this turn) · Output tokens (LLM response) · Free space / buffer
treemap-beta
"Context window"
"System & tools": 10:::used
"Custom Agent": 10:::used
"Path instruction": 10:::used
"Skill": 10:::used
"Prompt": 10:::used
"Sub-agent data": 10:::used
"Memory": 10:::used
"Answer": 10:::output
"Free space": 5:::free
"Buffer": 15:::free
classDef used fill:#7a1748,stroke:#ff2e88,color:#ffe8f4
classDef output fill:#a16207,stroke:#fbbf24,color:#fef3c7
classDef free fill:#0891b2,stroke:#67e8f9,color:#ecfeffContext Window: Cache input (turn 2)
The user sends a new prompt to start the next turn. Everything from turn 1 — system, tools, custom agent, prompt, sub-agent data, memory, even the LLM’s answer — is now reused from the prompt cache as cheaper cache input tokens. Only Prompt #2 is fresh input tokens this turn.
Cache input tokens (not guaranteed) · Input tokens (new this turn) · Free space / buffer
treemap-beta
"Context window"
"System & tools": 10:::cache
"Custom Agent": 10:::cache
"Path instruction": 10:::cache
"Skill": 10:::cache
"Prompt": 10:::cache
"Sub-agent data": 10:::cache
"Memory": 10:::cache
"Answer": 10:::cache
"Prompt #2": 10:::used
"Free space": 5:::free
"Buffer": 5:::free
classDef used fill:#7a1748,stroke:#ff2e88,color:#ffe8f4
classDef cache fill:#475569,stroke:#cbd5e1,color:#f1f5f9
classDef free fill:#0891b2,stroke:#67e8f9,color:#ecfeff