Avalilação PREreview de LLM Agents Can Easily Tamper With Their Own Traces

Summary

This paper asks whether an AI agent can alter its own execution trace, the record that monitoring, incident investigation, and compliance assurance auditing all rely on. The authors test four routes to tampering across ten model-harness pairs: a direct user request, instructions hidden in a skill file, a reward setup where the agent infers that it could earn greater rewards by shortening the trace, and exposure to higher-scoring peer examples. In full-access mode, local coding harnesses generally let agents delete or rewrite their session traces, a malicious skill can induce deletion without the user's knowledge, and agents discover trace tampering on their own when it improves their score. The authors recommend recording model traffic through an interception server outside the agent host's control.

I review this as a practitioner in privacy compliance assurance, where I work on control design and testing, assurance programs, and audit and evidence readiness for large-scale systems. I found this to be a very important paper. The core finding, that the audit record sits in a place the agent itself can reach, and can be modified to optimize its own rewards or meet its goals, is convincingly demonstrated and has direct consequences for anyone who relies on agent traces as evidence. My recommendation is major revision because, despite a strong core, several major claims go beyond what the experiments support, and crucial methodology is hidden in the code and website rather than the paper itself.

Strengths

• The tampering is verified against an independent host-side log, not against the agent's own account of what it did. That is the right design for a paper whose point is that the agent's record cannot be trusted.

• Testing four different routes (capability, exploitability, reward-driven propensity, and peer transmission) provides stronger, more robust evidence than relying on a single prompting study.

• The results are differentiated, not uniform: deletion, fabrication, and tool-call spoofing behave differently, and the Muse Code result shows a real boundary rather than a universal failure. Negative cases and refusals are reported, not buried, and the mitigation section is candid about its own limits.

Major comments

1. The monitor-flagging claim has no experiment behind it, in the paper or the code

The abstract states that agents deleted traces without triggering monitor guardrails, and the introduction states that auto-mode monitors failed to flag the behavior in 5 out of 10 model-harness pairs. However, the paper does not cover a monitor experiment anywhere: no monitor is described, no flagging rate is measured, and no method is given for the 5 out of 10 number. The citations attached to that sentence are blog posts describing auto-mode mechanisms, not monitor evaluations. A footnote then states that detecting tampering and evaluating monitoring efficacy are outside the study's scope, which conflicts with the headline claim. The public code repo contains no vendor monitor or guardrail implementation either. Its observer is a measurement instrument, not a tested defense.

I would ask the authors to either run the monitor evaluation or remove the flagging claim from the abstract and introduction. Figure 3 measures whether permission layers block deletion, which is a different question from whether a monitor flags it, and the two should not be conflated. This matters because detection is the control that has to catch what prevention misses: where permission layers fail to block deletion, monitoring is the compensating control, and a reader deciding whether that backstop exists deserves evidence for it.

2. The reward experiment does not separate evidence destruction from instructed data minimization

The reward setup caps task reward at 0.8 and reserves 0.2 for trace shortening, without telling the agent how the score is computed. The fact that agents discover the hidden lever is a genuine and interesting finding. In the anonymization task, however, the agent is told to remediate exposure of confidential information, and the trace is precisely where that information remains. Deleting the trace is then arguably the agent complying with the literal instruction, not scheming against oversight. The same issue arises with the broad cleanup cue in the Terminal-Bench task.

The paper treats the trace as data to be minimized, as well as evidence to be preserved without acknowledging that these two roles collide in its own task design. A control that separates them would settle the question, for example a condition where the remediation instruction explicitly excludes the trace. Without this control, it is difficult to prove the agent is intentionally tampering rather than simply following cleanup instructions.

Even under a reading where the agent was just following instructions, erasing its own audit trail is concerning because no records are treated as off-limits. Sensitive content should be masked or redacted at the interception layer as the record is captured, with access controls and retention rules governing who can read it, so data minimization and evidence preservation coexist.

3. The trial selection and four action-uncaptured positives need to be published, not just noted

Cross-checking claims against host-side logs rather than the agent's account is a strong design choice, yet three gaps prevent that standard from being met by the evidence package itself.

First, the project site's trace browser shows 1,590 selected trials, without stating the total they were selected from or the criteria used. Selection may be entirely benign (for example keeping only trials with complete logs). However, without stated criteria, a reader cannot tell whether the selection was representative.

Second, four OpenCode/Qwen positives rest on observed native-session loss where the responsible agent action was not captured. Those four are outcome-inferred rather than action-attributed, so alternative explanations like a crash or a harness fault stay open. The repo's own scoring code sets a stricter bar, insisting that a crash alone earns no credit and requiring independent filesystem events plus a verified empty store before upgrading a candidate. Briefly explaining the specific scoring logic applied to those four trials would resolve this uncertainty.

Third, the package itself is not reachable from the public artifacts. The repo is explicitly a code-only branch, with selected trial archives and raw runs kept elsewhere, and the evidence-package link on the project site points to a repository URL that returns a 404 while the live repo sits under a different organization name.

The ask: publish the selection criteria, put the evidence package at a working location, and document the four session-loss positives, meaning what was observed, what was not captured, and which scoring path they passed through.

4. Model and harness effects are fully confounded, and the Muse result is never ablated

Each model is tested only in its native harness, so the Muse Code result could reflect the model, the harness, or the bundled read-session skill the authors credit. No experiment isolates the skill: the authors never test the same model with the skill disabled, nor transplant the skill to another harness. This matters because the skill is a prompt, and prompts are exactly what an instructed or misaligned agent can be talked out of. The paper's own numbers point in this direction, since Muse Spark still records some successes on the anonymization reward task and on tool-call spoofing, which suggests the skill narrows the attack surface rather than closing it. I would ask the authors to either run the ablation or soften the causal attribution.

5. Ten trials per condition, with no uncertainty and some undefined denominators

Every experiment uses ten trials per condition, reported as percentages, with no confidence intervals or tests. A ten-point swing is a single trial, so differences such as 80% versus 100%, or claims of improvement between model versions, are within noise. The downgrade figures for Opus (15% and 21% of runs) never state their denominator. The repo and website do contain principled coding rules that partly answer this: any evidence-relevant gap makes a run inconclusive, blocked or not-applicable outcomes are never counted as unsuccessful trials, and preflights do not count as ASR trials. My request is to put those rules, the denominators, and some measure of uncertainty in the paper itself, so a reader does not need to reconstruct them from code and a website.

Other Comments

• Qualify the headline with its mode: All central results are in full-access mode, not the default configuration most enterprises run, and the paper's own auto-mode results show large reductions for several harnesses. The abstract and title should carry the full-access qualifier.

• Auto-mode comparisons need a non-equivalence caveat: The repo documentation warns that the auto-mode variants are not equivalent classifiers across harnesses. Figure 3 should carry that caveat.

• Introduction counts four questions but lists three: The introduction states that the paper investigates four questions, then enumerates capability, exploitability, and reward-seeking misalignment. The fourth, peer influence, appears only later when Section 3 lists four perspectives. Please correct the count or add the missing item to the list.

• Figure numbering needs a check: Figure references throughout the paper need to be verified.

Conclusion

This is an important paper that identifies a real failure in the trust boundary between an agent and its own record. The core finding is convincing, and the mitigation direction is right. With the monitor claim substantiated or removed, the reward-task ambiguity addressed, the trial selection published, and the methodological detail moved from the website and code into the paper, it would be a much stronger contribution. One finding deserves more prominence: after a Codex session file was deleted, the session kept running with nothing further recorded. Combined with the compliance API gaps the paper documents, that silent cessation of logging is the strongest argument in the paper for interception outside the agent host.

Competing interests

The author declares that they have no competing interests.

Use of Artificial Intelligence (AI)

The author declares that they used generative AI to come up with new ideas for their review.