Token Optimization

In a nutshell

Agent ROI

🚀 There are two ways to send a rocket to the moon: fire off many cheap ones and hope one lands, or carefully build a single high-quality rocket that hits on the first try. When tokens were cheap, spray-and-pray was fine — now, landing it in one shot is what actually costs less.

The compound-error problem

Even at 99% accuracy per step, a 50-step workflow degrades to roughly 60% final accuracy. LLMs are non-deterministic, so small per-step errors compound if you don’t catch them. The fastest way to save tokens is to spot small mistakes early and stop retries before they start.

Advice 1 — Provide as little context as possible…

In Context Engineering ↗ we saw how the window fills up turn after turn, and how an over-stuffed window leads to context rot — the agent gets dumber, not smarter.

• Short copilot-instructions — keep only the rules that truly apply every turn
• Don’t add a skill for everything — the model already knows React, TypeScript, Tailwind, etc.
• Short skills — a “caveman” skill can be replaced with one line (Be concise)
• Don’t spray path-instructions across every folder — they silently load on every call inside that folder
• Skip the “just in case” file preload — let the agent pull files when the task actually needs them

Advice 2 — …but as much as required

The other side of the lever. When context is missing, the agent fills the gap with assumptions — and those assumptions become the failed loops on the chart above.

• State the goal, the constraints, and the “done” criteria — vague requests cost more than verbose ones because the agent retries until it guesses what you meant
• Point at the right files instead of letting the agent grep blindly across the repo
• Surface non-obvious conventions — naming, error types, public-API rules, security boundaries
• Mention what NOT to touch — saves a wrong-file edit and a rollback retry later
• Paste the exact error or log line when debugging — saves a tool call (or three)

Advice 3 — Prompt engineering

💡 Your prompt is always-on — once sent, it stays in the window for the rest of the session, billed alongside system + tools on every turn. Make it count.

Advice 4 — Research → Plan → Implement

Divide and conquer your work. Don’t run research, planning, and implementation in one session — the window pollutes itself and the agent drifts. Split them into three sequential agents, each with the minimum context it needs.

💡 In Copilot CLI ↗, /research, /plan, and /fleet map directly onto these phases. Each phase runs in its own context window, so the bloated research session never reaches the implementer.

Advice 5 — Deterministic controls

Tests, linters, type checkers, and security scans aren’t just for humans — they’re the token-saving guardrails that stop one buggy change from snowballing into a four-step compounding spiral. With them, errors are caught on the same loop they were introduced. Without them, you pay in CI minutes, Copilot review cycles, and human triage.

📊 Evidence: the Copilot CLI team’s own codebase is over half tests.

Advice 6 — Model choice & Auto mode

Wrong tier = up to ~24× cost difference for the same task. Defaulting to a reasoning model for typo fixes is the most common waste. Pick deliberately — or let Auto pick for you.

Advice 7 — Tokenization is not language-neutral

The same sentence costs roughly 2~3× more tokens in Japanese than in English. For always-on text — copilot-instructions.md, skill descriptions, MCP tool descriptions, code comments — that gap is a tax billed on every single loop.

🔬 Verify it yourself: tiktokenizer.vercel.app ↗ or platform.openai.com/tokenizer ↗. Paste the same instruction in both languages and compare.

Advice 8 — Store knowledge in an agent-friendly format

Every .xlsx / .docx / .pdf you hand the agent triggers a 3-step detour: write a parser → run it → load the noisy output back into context. The parsed text typically runs 3–10× longer than the equivalent Markdown because layout metadata leaks through as noise.

Keep specs, knowledge bases, and reference tables in Markdown / CSV / plain text.

Advanced — power-user tips

These are conditional and come with trade-offs. Reach for them once the basics above are in place.

• 🖥️ CLI vs MCP — leaning on gh, kubectl, npm etc. is often leaner than the equivalent MCP because the model already knows these tools.
• ✂️ Trim shell output — tools like rtk-ai/rtk reduce LLM token consumption by 60–90% on common dev commands.
• 📊 Run /chronicle tip regularly — analyze your Copilot CLI sessions to surface concrete improvement areas.
• 🔁 Collapse tool calls — copilot-codeact-plugin batches multiple tool calls into one round-trip.
• 🎚️ Model-specific tuning — possible, but models change fast; only worthwhile at very high scale.
• ⌨️ Ctrl+I in the VS Code terminal — a one-off question on the command line is ~10–30× cheaper than a normal session on the same Auto model, because almost no context is loaded. Ideal for quick checks.

The long-term mindset

• 🧭 Build analytical skills. Coding was never the true value of developers — analytical skills and domain fluency were. Telling an agent precisely what to do, in the speak of the domain, becomes the most valuable craft.
• 🏛️ Apply good architecture. Domain-Driven Design, Hexagonal, CQRS, Event-Driven — clean boundaries give agents stronger guard rails and prevent them from putting code in the wrong place. Debates about 5-line functions matter less than ever; architecture matters more.
• 🔧 Iterate on prompts & configs. You’re a context engineer now. Treat agent misses like incidents, keep configs fresh, and use /chronicle to spot patterns.

8 things to start doing today

1. ✅ Provide as little context as possible. Trim your instruction files, skills, and custom agents — and write them manually.
2. ✅ But provide as much as required. Give it the spec, examples, and constraints it needs to one-shot the task.
3. ✅ Engineer your prompts. Be explicit about goal, output format, and constraints.
4. ✅ Research → Plan → Implement. Split the three phases into separate sessions or subagents, each with its own context window and the right model.
5. ✅ Provide deterministic controls. Tests, linters, type checkers, and security scans catch errors on the same loop they were introduced.
6. ✅ Pick the right model — or let Auto pick. Reasoning for planning, Mid for implementation, Low for chores.
7. ✅ Write your harness in English when the team can read it. Tokenizers still cost ~2–3× more on Japanese.
8. ✅ Store data in Markdown for agents. Binary formats (xlsx, docx, pdf) waste 3–10× the tokens on parsing-tool calls every time the agent reads them.