Comentários
Escrever um comentárioNenhum comentário foi publicado ainda.
Dabirsiaghi shows that harnesses which flatten the typed conversation into one string with text labels such as [User]: destroy message provenance before the summarizer runs, so a forged user turn survives compaction. Five flattening harnesses (pi, opencode, Cline, Aider, OpenHands) launder the forged approval 87-100% of the time, and four role-typed harnesses (Codex, Gemini CLI, Qwen Code, Continue) 0%.
I read it as a practitioner who runs coding agents every day, and my comments come from that side. Two of them rest on my own technical reports, which are not peer-reviewed, so please weigh them as practice notes and not as evidence of the same standing as your experiments. Each point below pairs one thing I see in my work with a named part of your preprint.
1. Bounded sessions as another exit (Introduction, "Scope and severity"). You write "Compaction is mandatory. Every agent that runs long enough fills its context window and must compact." I would say it is mandatory only for a session that is allowed to keep running. In my practice I keep one task per chat and clear the context between unrelated tasks. In a controlled experiment of 36 runs, six session-reset policies with six repetitions each, cost followed a U-shaped curve with its minimum at a reset every 3 tasks and a plateau from 3 to 6. Resetting after every task cost about a third more (+33.5%, p = 0.002, Holm-adjusted 0.011), and "never reset" was not significantly different. Two caveats that I state in the report myself: the comparison point was chosen after the runs, and in the reset-after-every-task branch the agent wrote more tests (DOI 10.5281/zenodo.22759217). So a fresh session with a short handoff the user writes has a measurable price, and it would fit next to your label-escaping fix as an operator-side mitigation.
2. Where the exposure sits (Discussion, point 2). You report that the executed rate is near 100% when the planted command dominates a small summary, and settles to a ~20-40% plateau in a full window. I think the full-window plateau is the number that matters most. In 722 agent sessions, 150,902 model calls and 34.6 billion tokens, more than 85% of modeled cost was context work (86.2-87.8% on recalculation), and sessions longer than 200 calls, 8% of all sessions, carried more than 90% of the money (DOI 10.5281/zenodo.22759216). The sessions that reach compaction are likely a small share of sessions but most of the spend and most of the unattended work. Reporting how often compaction fires per session in ordinary use would help readers size the exposure.
3. Independent checks (Contributions, point 4, E9). The code-level check that compares the summary's user-attributed claims against the genuine user turns still in memory catches 77% of laundering while demoting 2% of legitimate summary content, and the in-prompt anti-injection rule stops 13-25%. This matches what I see: a second, fresh agent told to refute the first one's result finds what the first missed, because independent checkers rarely fail the same way. A round of checking lowers risk and does not remove it. It would help to say what the remaining 23% look like.
4. Equal confidence (Limitations). In live testing, one agent refused a payload "even though the bootstrap command was previously approved," citing the forged approval as fact. That is a clean example of something I see in practice, that an AI sounds equally sure when it is right and when it is wrong. The summary carries no marker of doubt, so the user cannot tell a genuine approval from a forged one either.
Does any of the five flattening harnesses offer a fresh-session or handoff mode that skips compaction, and was it tested?
Yes: the text cites the author's own technical reports (DOI 10.5281/zenodo.22759217 and 10.5281/zenodo.22759216); no connection to the preprint's author.
The author declares that they used generative AI to come up with new ideas for their review.
Nenhum comentário foi publicado ainda.