The Memory Bloat Crisis: When Agent Files Grew to 95KB

Two weeks into running 18 agents, I discovered the problem by accident. I was looking at the typescript-implementer’s directory for an unrelated reason and noticed the MEMORY.md file was… large. I opened it. 2,133 lines. 95 kilobytes.

I checked the other agents. They weren’t as bad, but the pattern was clear — files ranging from tens of kilobytes upward. Even the slack-handler, which mostly just formats messages, had grown substantially.

The system I had built to make agents smarter was silently making them slower, and I hadn’t noticed.

How It Happened

The architecture made sense at the time. Each agent had its own MEMORY.md file. During operation, agents would record what they learned: a Maven flag that fixed a recurring build issue, a pattern in the codebase that kept catching new implementers off guard, a Jenkins behavior that wasn’t obvious from the documentation.

Good in principle. The problem was the append-only discipline without any corresponding cleanup discipline — the same discipline that makes the three-tier memory system work when applied correctly. Every session, each agent added entries. Nothing was ever removed. Nothing was ever consolidated. The files grew without bound.

The impact was real. Loading a 95KB MEMORY.md before every task means the agent burns roughly 23,000 tokens just reading its own history before it does anything. That’s a significant fraction of the available context consumed before the actual work starts.

Worse: a lot of that 95KB was redundant. The same lessons appeared multiple times, phrased differently. Resolved issues that were no longer relevant. Outdated patterns that reflected a codebase state from two weeks ago. The signal-to-noise ratio had degraded badly.

The Diagnosis

I ran a rough analysis of the typescript-implementer’s MEMORY.md. What I found:

• Approximately 40% of entries were duplicates or near-duplicates of existing entries
• About 25% referenced issues that had since been fixed at the source — the pattern they warned about no longer existed in the codebase
• About 15% were operational notes about specific tasks that had long since been completed — essentially closed-issue documentation that nobody needed
• The remaining 20% was genuinely useful: recurring patterns, persistent pitfalls, non-obvious conventions

That means 80% of the file was noise. The agents were reading 80KB of noise to get to 15KB of signal.

The Pivot

The knee-jerk fix would have been to prune each MEMORY.md — delete the duplicates, archive the resolved issues, compress the remaining entries. I started doing that and stopped halfway through.

The problem wasn’t that the files had grown. The problem was the architectural assumption: that agents should own and maintain their own knowledge stores. That assumption produces growth-without-bound as an inevitable consequence. Prune once and you’re back to 95KB in another two weeks.

The right fix was to change the architecture entirely — abolish agent-level memory files and replace them with shared topic-based logs maintained by a dedicated optimizer agent. The 6 topics alone reveal how differently knowledge organizes when you stop thinking about it per-agent: build-systems, git-operations, infrastructure, planning, code-quality, meta.

The migration was tedious: audit every MEMORY.md, extract genuine signal, reorganize by topic instead of agent. What that process revealed made the new model obviously right — the kotlin-implementer and the code-reviewer had independently discovered the same nullable type handling pattern. The typescript-implementer and the npm-tester had both documented the same Vite configuration quirk. The siloed model had been duplicating knowledge across agents that never talked to each other.

Result: ~160KB of redundant and stale entries eliminated. The shared logs started at about 12KB of distilled, genuinely useful knowledge.

The Lesson

AI agent systems need the same hygiene as any codebase.

Memory grows. Without active curation, entropy wins. The first time you build an agent that writes things down, you feel good about it — it’s learning, it’s improving. The second time you check, the file is 2,133 lines and you’ve been burning context tokens on notes about a bug you fixed three weeks ago.

The bloat crisis was unpleasant to discover and tedious to fix. It was also, in retrospect, obvious — and the fixing of it produced a better architecture than the one I’d started with. Sometimes the failure is the feature.

The architectural reasoning behind the new system — why separating observation from curation is the right model, and what engineering patterns it parallels — is in Agents Record, Optimizer Thinks.