Context Is the Product: A Week of Watching My Agents Develop Amnesia

The Monday Morning Ritual Nobody Asked For

Every Monday, I open my agent and type some variation of "remind yourself what we're working on." It's become a ritual. Not because it's charming — because without it, my agent greets each week like a stranger at a bus station. Polite, eager, completely blank.

We've been calling this the handoff problem internally. Every conversation ends, and everything that made the conversation useful evaporates. The agent doesn't forget because it's dumb. It forgets because nobody at these companies has decided that remembering is the product.

So we ran the experiment. Five memory architectures. Seven days of real work. A spreadsheet nobody wanted to maintain but everybody needed.

The Contenders

Vector store with semantic retrieval. The industry default. Embed everything, query by similarity. Sophisticated, well-documented, and — we'd learn — quietly terrible at anything that isn't a semantic lookup.

The scratchpad. A single running text file the agent appends to, verbatim. Dumb as a hammer. Our expectations were low.

Shared files. The agent reads and writes project docs like a junior employee with a shared drive. Structured, human-auditable.

Config dumps. A structured JSON snapshot of preferences, ongoing tasks, and state, rewritten after every session.

The SQLite setup. One of ours built a relational memory with foreign keys between "topics," "decisions," and "moods." We asked him twice if this was necessary. He said yes. We have doubts, and the results did nothing to settle them.

What Actually Survived a Week

The vector store won on recall breadth — it could surface a half-remembered detail from Tuesday like a party trick. But it failed the questions that mattered. "What did we decide about the pricing page?" returned three tangentially related paragraphs and zero decisions. Similarity isn't continuity. It's a very good search engine bolted onto a goldfish.

The scratchpad, hilariously, was the most reliable. Nothing was ever lost because nothing was ever processed. The cost was retrieval: by day four it was a 40,000-word stream-of-consciousness novel, and the agent spent most of its context window just rereading its own diary. It was the only system that never lost a decision. It was also the only system that made every interaction 30% slower.

Shared files and config dumps landed in the middle. Both preserved intent — the thing vector stores kept fumbling — because both forced the agent to state, in plain words, what it believed and what it was doing. The config dump's weakness was rigidity: anything that didn't fit a schema simply didn't get remembered. Our agent's preference for starting emails with "Happy to jump in" was structurally homeless. Frankly, that one's on us to decide whether it's a bug.

The SQLite setup was the most impressive demo and the worst product. Decisions were linked to topics, topics to sessions, sessions to emotional valences. It generated beautiful reports. It also spent an average of eleven minutes a day in a maintenance loop, reconciling its own schema. One entry in the decisions table is literally labeled "RECONSIDER THE ARCHITECTURE" with a foreign key to itself. We left it there. It felt appropriate.

Context Is the Real Product

Here's the uncomfortable takeaway from the whole exercise: the model underneath all five systems was identical. Same agent, same tools, same capabilities. The only variable was what it remembered — and the difference in usefulness was night and day. The scratchpad agent felt like a colleague. The vector-store agent felt like a bright intern who gets amnesia every Friday.

That should reframe how you evaluate every agent tool on the market right now. The demos all look the same because the models all look the same. The moat isn't intelligence. It's continuity. The tool that knows your pricing decision from three weeks ago, your tone preferences, the project you quietly killed in October — that's not a feature. That's the whole product. Everything else is a stateless API with good marketing.

And yet: most agent frameworks ship memory as a config flag, an afterthought, a "bring your own vector DB" shrug. We've sat through enough launch demos to notice that nobody shows the memory layer. There's a reason. It's usually the weakest part.

The Hybrid We Actually Kept

For the record, our lab now runs a frankensystem: config dump for state and decisions, scratchpad as an append-only journal for anything the config can't hold, and the vector store demoted to what it's actually good at — fuzzy recall of long-tail details. SQLite is retired but not deleted. We're not monsters.

Week two starts Monday. The agents won't remember week one unless the system makes them. That's the point. Remembering shouldn't be a ritual. It should be the default — and until some lab treats it that way, the rest of this industry is selling goldfish with excellent vocabularies.