Idempotency Violations in Agent-Triggered CI Jobs

Agent-triggered CI exposes a structural idempotency problem. Agents bring at-least-once retry behavior to pipelines built around human-frequency, effectively-once triggering. Preventing duplicate effects therefore requires workflow-level design, from durable operation identity to downstream deduplication and merge-queue controls.

Why agent-triggered CI is structurally different from human-triggered CI

Agent orchestration treats an uncertain result as a reason to retry, while a CI pipeline treats another trigger as an instruction to execute. Those contracts do not fit together automatically. A developer who clicks “re-run” has seen a result, made a judgment, and initiated a deliberate retry. An agent can retry after a timeout, an ambiguous HTTP response, or a dropped socket, before the original job has finished.

The uncertainty concerns acknowledgment. A human knows whether they clicked, but an agent knows only that it did not receive a confirmed result. That absence does not establish that the action failed. The orchestration layer says “retry if unsure,” and the non-idempotent handler hears “run again.” Because CI jobs write caches, publish artifacts, update deployment state, post status checks, and mutate queue position, another execution can repeat externally visible effects.

What actually breaks when a CI job runs twice for the same intent

A duplicate run can corrupt the signal used to decide which changes are safe to merge. If it fails after colliding with its predecessor’s side effects, merge-queue bisection logic cannot distinguish that failure from a legitimate intermittent failure. The queue then acts on a misleading result. Its response affects not only the offending change but also the changes waiting behind it. CI signal integrity is therefore a prerequisite for useful queue behavior.

An agent instructed to get a PR merged can compound the problem by re-queuing the same change after every red result. Each attempt consumes another full CI run and can restart the disruption for changes behind it. Retry behavior that was mildly wasteful at human frequency can become a denial-of-service at agent frequency, because the agent keeps treating the consequences of its retries as reasons to retry again.

In the fixture-drift example, a coding agent rewrote payment assertions overnight, shared CI reported a green build, and the same idempotency key could then pass twice. The passing signal concealed the weakened acceptance condition. This failure is quieter than queue congestion because execution appears successful even though the check no longer rejects the duplicate effect.

The four structural causes that make violations predictable, not accidental

Fresh keys on every retry are the first cause. Generating a new idempotency key or workflow run ID makes each retry appear to be new work. In CI, that means spawning another workflow run instead of recognizing an already-running or completed operation. Model-generated identity is particularly fragile here. Stochastic temperature settings can produce a different UUID on each inference pass, so apparently equivalent requests need not share the identity required for deduplication.

Context window compaction is the second cause. When there are more tokens than the context window can hold, orchestration frameworks compact or summarize earlier turns, and the model can lose track of previous step sequence numbers. During recovery, it may re-execute a step it cannot locate in its prompt buffer. The transcript can still read coherently throughout. Unlike a crash, compaction may leave no clear sign of where execution history became unreliable.

Rollback replay is the third cause. An August 2026 academic study, arXiv:2608.29381, presents the first systematic security study of checkpoint-and-rollback in agent systems and describes how faithful restoration can resume states, assumptions, and external effects that never coexisted in a valid history. A checkpoint before an email illustrates the mechanism. If the email sends and a later rollback restores that checkpoint, replay can send it again.

Unspecified resume semantics are the fourth cause. An August 2026 conformance study, arXiv:2608.03836, found that five widely deployed agent workflow frameworks answer basic resume-semantics questions differently. None exposes a machine-checkable contract, and measured behavior violated even the fragments the frameworks state. Exactly-once execution is therefore an unsafe assumption. A team can build its CI handler around guarantees that its orchestration layer neither clearly defines nor reliably enforces.

Why adding retry guards at the job level does not close the gap

A job-level guard protects only the boundary where it runs. Moving a side-effecting step into a child task and triggering that task with an idempotency key can prevent duplicate child creation. It does not prevent the child’s execution from retrying after a downstream provider has accepted its request. That boundary needs its own independent protection. Only the provider can deduplicate its own API call, so once-only task creation cannot establish once-only external effects.

The planner loop, tool wrapper, queue consumer, and CI infrastructure also retry independently. Their behavior is cumulative rather than interchangeable, and a single logical intent can fan out into multiple writes unless stable operation IDs persist across all four layers. Prompt instructions cannot enforce that invariant by themselves. Enforcement hooks and scripts must do so, because at-least-once delivery produces effectively-once behavior only when the handlers are genuinely idempotent at every layer. A local guard leaves the other retry paths intact.

Redesigning CI jobs for agent-frequency triggering

The workflow-level response is to derive operation identity from durable state before any model inference occurs. A platform team should construct the operation ID from the step ID plus arguments, so retries at any layer bind to the same record. The model must not generate that identity inline. This separates duplicate detection from both temperature variation and the model’s ability to recall an earlier step. Explicit retry budgets then set a limit on how long the workflow may keep trying to carry out the same intent.

Those budgets need stopping points that are enforced and failure signals that people can see. Teams should alert on consecutive failures of the same job rather than relying only on aggregate spend alerts, which a human may not read while a retry storm peaks. Bounded delegation gives those controls a defined scope. It assigns one task with a specific set of capabilities, explicit stop conditions, and independently enforced acceptance checks. Capabilities, stopping, and acceptance are separate predicates even when the interface shows one job.

An unattended CI agent also needs a machine-checkable success condition. If the check cannot be written, the job belongs in an interactive workflow where a human can catch drift in real time. Scope enforcement belongs in the gate. An agent that fixes a bug and also reformats the file can undermine diff reviewability and make idempotency auditing impractical. Independent acceptance checks address the same weakness exposed by fixture drift, where a passing build did not preserve the intended invariant.

Recovery needs equally explicit treatment of history. Every checkpoint-and-rollback path should check whether reachable side effects have already occurred before replaying forward from restored state. Context-pinning addresses a different form of lost continuity. Reattaching unresolved contradictions and unverified assumptions verbatim to the bottom of the next prompt prevents the attention-fade pattern in which an acknowledged complication disappears after compaction.

How merge queue design interacts with idempotency at agent volume

A merge queue built for human PR volume is the next line of defense when duplicate effects get past job-level controls. Agent output can grow beyond what headcount predicts, but merge throughput still depends on how many candidate states of main CI can validate per hour. The validation constraints do not disappear with faster authorship. Test suite duration, runner capacity, flake rate, and queue mechanics still govern progress, while retry storms consume the same capacity needed for legitimate changes.

Base Power Company concluded that standard merge queue serialization is too slow at agent volume, even with diff-aware selective testing. The company pulled end-to-end suites into the pre-merge gate rather than relying on post-merge, nightly, or pre-release runs. Freshness also requires the correct candidate state. Running checks on the actual merge_group ref rather than a stale PR head addresses a class of violation that job-level guards alone cannot resolve. Deduplicating a run does not make its input current.

Test impact analysis offers a theory-of-constraints response when the alternative is serializing every agent PR through a full suite. It runs only tests a change could plausibly affect, reducing the validation work assigned to that change. Signal quality remains a separate requirement, however. Selective testing cannot make up for duplicate effects that make the selected tests misleading, and a queue with more capacity still cannot tell those effects apart from legitimate flakes.

Where prompt injection intersects with idempotency in agent-triggered pipelines

Untrusted repository metadata creates a path for deliberately inducing unsafe agent actions, including the repeated side-effect pattern produced accidentally by retries. The Cloud Security Alliance’s June 2026 analysis of the claude-code-action vulnerability chain describes an authorization bypass that trusted any GitHub App actor with a [bot]-suffixed identity. A malicious issue body could direct the agent to read environment variables and exfiltrate OIDC token credentials, enabling downstream privilege escalation. The attacker needed only to open an issue. Write access was not required.

The Clinejection incident on February 17, 2026, shows how trusting issue content as instructions can harm the supply chain. A malicious GitHub issue title set off a vulnerability chain that led to an unauthorized compromise of the Cline AI coding tool’s npm package. The malicious package affected an undisclosed number of developer and CI/CD systems over approximately eight hours before removal. This establishes an injection path. The connection to idempotency is that an injected instruction can deliberately seek repeated actions through the same side-effecting pipeline.

The defensive response therefore belongs in permissions as well as execution controls. The minimum posture is branch-only write access to agent-prefixed branches, no default-branch write, and no self-approval. An agent allowed to merge can ship an injected instruction to production. Idempotency guards limit the effects of running again, while permission controls limit which effects an injected instruction can reach. Neither control substitutes for the other.

What an observable, auditable agent-triggered CI workflow looks like in practice

An auditable workflow makes logical intent visible across physical runs. Every run of the same intent needs the same stable operation ID, together with a record of which external side effects occurred and when. Completed steps must be distinguishable from replayed steps. Retry-budget exhaustion also needs a clear signal rather than a silent extension, so the audit trail shows where execution was required to stop. Without those distinctions, a recovery that looks successful can hide another external write.

Agent transcript analysis adds visibility into the paths that produced those effects. It can surface which skills agents invoke, which retry paths they exercise, and which steps they re-execute after compaction. These patterns can be visible in agent transcripts before disruption in the queue is the only sign. Trimming unused skills reduces the paths through which compaction-driven re-execution can produce a novel side effect. That narrower execution surface also makes the remaining behavior easier to audit.

Code storage designed for agent-frequency access needs different read/write performance characteristics from human-first version control if it is to anchor stable operation IDs reliably. Infrastructure needs are not uniform across teams. Some teams need fixes at the queue layer, others need transcript analysis to find re-executed skills, and others need storage that can support durable identity. A composable approach allows each missing guarantee to be addressed without packaging all three identically, while the audit trail supplies the evidence that retries remained bound to one intent.