Context Window Management
Context window management is one of the most overlooked PM skills for working with AI. Most degradation in output quality — the model "forgetting" earlier instructions, contradicting itself, losing nuance — traces back to context problems, not model capability. This skill diagnoses context issues and provides practical techniques to fix them.
---
Context
What a context window is (PM-level explanation):Every AI model has a maximum amount of text it can hold in working memory at once — this is the context window. Everything you've said, everything it's responded with, and everything you've pasted in counts toward that limit. When the window fills up, older content gets dropped. The model doesn't tell you this — it just starts producing worse output.
Current context window sizes (as of 2025):| Model | Context window | Practical usable limit |
|---|---|---|
| GPT-4o | 128k tokens | ~80k tokens before quality drops |
| Claude 3.5 Sonnet | 200k tokens | ~150k tokens |
| Gemini 1.5 Pro | 1M tokens | ~600k tokens |
---
Step 1 — Diagnose the context problem
Classify the issue: Context overflow, Input too large, Attention diffusion, Cross-session forgetting, or Irrelevant context pollution.
Step 2 — Apply the right technique
Technique 1 — Structured context front-loading (for attention diffusion)
Critical instructions at start and end. Explicit section markers.
Technique 2 — Chunking (for input too large)
Process at natural boundaries. Carry forward a running summary, not the full chunk.
Technique 3 — Context compression (for context overflow)
Summarise the session into a context brief and start fresh.
Technique 4 — Selective context inclusion (for irrelevant context pollution)
Fresh session with only the relevant context. Exclude earlier conversation turns.
Technique 5 — External memory (for cross-session forgetting)
Maintain a living document with standing context, current state, key decisions, and "do not revisit" list. Paste standing context at start of every session.
Step 3 — Output the context management plan
Include: diagnosed problem, recommended techniques, and rules going forward (session length limit, chunk size, external memory decision, fresh session trigger).
Quick reference — context window red flags
| Red flag | What's happening | Fix |
|---|---|---|
| Model contradicts earlier instruction | Instruction dropped from window | Compress + fresh session |
| Model answers different question | Attention diffusion | Restructure prompt |
| Model stops following format | Format instruction lost | Restate format at end of every prompt |
| Model references wrong document | Context pollution | Fresh session with selective context |
| Model becomes more generic over time | Window filling with noise | Compress context; remove irrelevant turns |