Reading notes
Papers that changed the system
Not a canon. Each of these changed a concrete decision in the agent platform I build, and the note says what it changed. The set grows as the system does.
Persist an agent's knowledge as an append-only event log rather than conversation state.
Independent convergence on the design I'd already bet on: our fact store is event-sourced, every fact carries provenance, and the case is replayable from the log. When the agent re-derives what it already knew, you don't need a smarter model. You need state below the model.
Automate SOPs by giving the agent a global repository of actions and procedures it retrieves from, rather than inlining everything.
Medical billing is SOPs all the way down: appeals have playbooks, payers have procedures. The registry pattern here is close to our context engine, and it confirmed the sharpest lesson of that rewrite: a new capability should be a file the agent can read, not surgery on a prompt.
Structure prompts as named, ordered routines with explicit steps instead of prose instructions.
Half of “prompt engineering” is actually software engineering under a different name. Naming and ordering routines is modularization; this paper just applies it inside the prompt. We went further and moved the routines out of the prompt entirely, but the instinct is the same.
Practical eval methodology for agents: score whole trajectories, use pass^k, start from real failures rather than generic benchmarks.
The most useful writing on evals I've found because it starts where teams actually are: a pile of anecdotes and no harness. Our regression suite grew exactly this way, seeding eval cases from production failures, so a prompt change that regresses last month's cases fails loudly.