Escribir un comentario

This essay argues that the practices long recommended for reproducible research software — an agent instruction file, characterization tests, a descriptive commit history, a predictable repository layout and a decision record — are now also context engineering for AI coding agents, and that agents make these artifacts cheap to produce while the researcher stays their owner and verifier. I read it as a practitioner who runs coding agents every day and measures what their context costs. The argument matches my experience, and I have four observations from my own published measurements and practice that bear on it.

1. Always-on context has a recurring price. The essay describes the agent instruction file as the document an agent reads at the start of every session (Section 2.1). In practice that file travels with the conversation, so it is paid for again and again rather than once. In my analysis of 722 agent sessions and 150,902 model calls, context handling accounted for 83.5% of modeled cost and generation for only 16.5% (Arsentev, 2026, doi:10.5281/zenodo.22688706; technical report, not peer-reviewed). That is a practical reason to keep instruction files short. My rule is to keep such a file under about 200 lines and to test every line by asking whether removing it would make the agent start making mistakes; if not, the line goes. A bloated file is followed worse, not better. A sentence on length would make the catalog in Section 2.1 easier to act on.

2. When a session ends is also a context decision. The essay places tests, commits, structure and decision records at "the timescale of a codebase rather than the timescale of a conversation". Between the two sits the choice of how many tasks an agent handles before its context is cleared. In a controlled experiment with 36 runs under six session-length policies, clearing after every task cost 25.3% more than clearing after every third task, and never clearing cost 15.6% more, while three, four and six tasks per session could not be told apart (Arsentev, 2026, doi:10.5281/zenodo.22699668; technical report, not peer-reviewed). The instruction file is exactly what lets a fresh session start already knowing the project's rules, so the artifacts in the catalog and deliberate session resets work together.

3. Verification. I agree that verification is the one task that cannot be handed to an agent (Section 5). A layer I use before the human check is a second, fresh agent whose only instruction is to try to refute the first agent's result and to treat it as wrong until shown otherwise. Independent agents rarely make the same mistake, so a disagreement points straight at the weak spot. This lowers risk rather than removing it, and anything consequential still needs the human review the essay insists on.

4. Every task with its own check. Section 2.1 describes the test suite as the signal an agent runs to verify its own work. That is the habit I recommend most often: give every task a check the agent can run itself, because without one, "looks done" is the only signal the agent has.

Competing interests

The review cites my own technical reports on the cost of context in coding-agent sessions (doi:10.5281/zenodo.22688706 and doi:10.5281/zenodo.22699668). I have no connection to the author of the preprint.

Use of Artificial Intelligence (AI)

The author declares that they did not use generative AI to come up with new ideas for their review.