CI/CD Pipeline Design for High-Volume Agent PR Output

CI/CD pipelines built for human developers cannot absorb the volume, speed, and concurrency of agent-generated pull requests, and no amount of configuration tuning closes that gap. Ability.ai's 2026 analysis names the break directly: pipelines built around one or two diffs a week cannot handle thousands of concurrent machine-generated commits, and the resulting failure is an architectural mismatch rather than a performance gap.

The pull request itself was designed as a discrete handoff point for delayed human feedback. Forcing an agent to sit in a queue built for human-paced review neutralizes the very capability the agent was built to deliver: speed.

The capability curve driving this problem moved fast enough that the infrastructure never had a chance to catch up. Performance on the SWE-bench Verified benchmark, a standard measure of real-world GitHub issue resolution, climbed dramatically over a short window, and that compressed leap in agent capability has already outrun the infrastructure built to receive its output. Agents did not invent new weaknesses in CI/CD systems. They found the latent ones that human-paced development never stressed hard enough to expose: the serialized merge path, the shared runner pool, the test suite that tolerates occasional flakiness because occasional is all a human cadence ever produced.

That is the shape of the problem this piece works through. The constraint does not disappear when agents produce more code. It moves downstream, concentrating at the narrowest point in the system: the serialized path between a PR marked "approved" and a PR that is actually sitting on main. Every structural fix that follows, queue design, governance logic, test execution, compute scaling, follows from that single relocation of the bottleneck.

Queueing dynamics that turn moderate agent adoption into a full pipeline stall

Queueing math produces this failure: as agent PR arrival rates approach the throughput capacity of the CI system, wait times do not creep upward, they explode, and the queue becomes the dominant bottleneck well before any test ever flakes. That is the nature of queues operating near saturation: small increases in arrival rate produce disproportionate increases in wait time, and a team that feels like it is coping can find itself in a full stall.

Consider the arithmetic of a single merge queue. If a merge-group CI run takes a fixed end-to-end duration, a strictly ordered queue can land only a small number of PRs per hour, and that ceiling holds to perhaps a few dozen per day even when nothing fails. Agents do not politely stay under that ceiling. They produce in bursts, and the queue has no mechanism for distinguishing a burst of ten agent-generated PRs from the one or two a human team would have produced in the same window.

This pattern appears clearly in adoption data. Teams with high AI adoption merge nearly twice as many pull requests as they did before, yet their delivery metrics barely move, because every one of those additional PRs still has to squeeze through the same review pipeline, the same CI fleet, and the same merge queue that was sized for human typing speed. Agents write more code. The pipe downstream of that code did not get wider.

Flaky tests turn this into something closer to collapse. A single red test invalidates the speculative state that every PR behind it in the queue was validated against, forcing the whole queue to restart from the point of failure. This is not a hypothetical tax. Before Uber built dedicated flake management infrastructure, its iOS mainline was green only about half the time, a flaky-test rate rather than a reflection of code quality.

Retry loops that were mildly wasteful at human scale, where a developer might re-submit a failed PR once or twice a day, become something far more damaging at agent frequency: an agent instructed simply to "get the PR merged" will re-queue the same change repeatedly, burning a full CI run on every attempt and re-triggering the invalidation cascade for every PR behind it.

Runner starvation compounds the queueing pressure further, and the failure mode is not theoretical. Shared runner pools without scheduling logic that accounts for queue position will invert priority exactly this way, letting PRs that arrived later consume the capacity that PRs at the front of the line are waiting on.

Designing a merge queue for agent throughput

Adding more queue slots does not touch any of the structural causes identified above. It only delays the point at which the same failure modes reassert themselves at a slightly higher volume.

Footprint-aware parallelism starts from a simple computation: each change carries a machine-computed answer to what it actually touches in the codebase. When two PRs' footprints are disjoint, they modify no overlapping files or systems, so those changes commute with each other and can be validated concurrently in separate lanes, with serialization reserved only for changes whose footprints genuinely intersect. Base Power Company built this kind of lane logic in direct response to its most prolific contributors, who were submitting a high volume of PRs weekly through parallel coding agents that pushed, failed, fixed, and delivered entire stacks of PRs with passing CI checks. The pattern is instructive regardless of which team implements it: a queue that can prove two changes do not touch each other has no structural reason to serialize them.

Stack-aware queuing solves a related but distinct problem. Processing an entire stack of dependent PRs together, rather than one PR at a time, avoids the artificial serialization that occurs when each PR in a stack has to wait for the PR above it to land before its own validation can even begin. The Base Power and Graphite examples illustrate what the design pattern looks like in practice rather than serving as endorsements of either product.

The third principle, agent-operated conflict and failure handling, changes who resolves problems once they surface in the queue. When a merge conflict or a CI failure occurs, the resolution step itself should go to an agent rather than blocking the entire queue while a human gets paged. This is the merge queue's own version of the shift from human-paced to machine-paced operation: a queue that stops and waits for a human on every mechanical rebase conflict is not built for agent-scale throughput, regardless of how parallel its lanes are. Human attention should be reserved for decisions that actually require judgment, ambiguous conflicts with architectural implications, changes flagged for security review, or changes touching production configuration, not for conflicts a script could resolve.

Queue-position-aware runner scheduling has to accompany these architectural changes, not trail behind them. Keeping the footprint computation conservative, a straightforward union of touched files rather than an inferred semantic judgment about what might interact, and pairing it with a fast sanity check immediately after each merge, before the next batch is allowed to begin, is the answer.

Governance of agent PR output, built into the queue, not bolted on after

Agent-generated PRs carry quality and duplication patterns that a queue optimized purely for throughput will amplify rather than catch, which makes governance logic a structural requirement for the queue rather than a policy layered on top of it afterward.

The security dimension of this is not hypothetical. An unauthorized package version was live on npm for roughly eight hours as a result. Agents with shell access to CI runners, combined with unsanitized external inputs, constitute an attack surface, and treating them only as a reliability concern misses what actually went wrong.

The quality asymmetry behind this incident is measurable across agent output generally. Cross-tool studies find that AI-co-authored PRs contain substantially more issues than PRs written entirely by humans, and security-related agentic PRs show both lower merge rates and longer review latency, reflecting the heightened scrutiny human reviewers apply to them. The throughput gains a well-designed merge queue delivers are not distributed evenly across risk levels, and a queue that treats every PR identically regardless of what it touches will route security-sensitive changes through the same fast lane as a formatting fix.

Duplication is the second governance failure that throughput optimization alone will not catch. A CNCF project documented this at production scale in 2026: a fleet of agents was producing fixes faster than the queue could land them, and as of August 8, 2026, the project had 96 open PRs fleet-wide, 94 of them mergeable, with roughly a quarter of that open queue redundant because no mechanism existed for cross-agent deduplication. If the queue itself has no deduplication logic, each agent independently opens the same fix for the same problem, and the queue spends CI capacity validating changes that are functionally identical to ones already in flight. Governance documents describing what agents should and should not do do not substitute for an operating system that actually enforces those rules at the point where PRs enter the queue.

The structural response to both the security and duplication problems is a trust-tier model. The AI-Augmented CI/CD reference architecture proposes staged autonomy, in which agents operate as policy-bounded co-pilots inside guardrails defined through policy-as-code, using Open Policy Agent, with human override reserved for cases where confidence thresholds are not met. Under this model, different categories of change earn different levels of autonomy: low-risk formatting and test changes can be auto-merged outright, while changes that touch security logic or production configuration require human approval regardless of what the CI status shows. This is the governance equivalent of footprint-aware parallelism discussed in the previous section. Just as that architecture routes changes by whether their footprints intersect, the trust-tier model routes changes by risk class rather than by the order in which they arrived in the queue.

Rethinking test execution so flaky tests stop setting the throughput ceiling

Flaky tests are not a quality issue a team manages around at the margins. At agent PR volume, they become the primary ceiling on throughput, and the only durable response is to treat flakiness detection and isolation as first-class infrastructure built into the pipeline itself, built in before a team is annoyed enough to demand it.

The cascade mechanism explains why the cost compounds so badly. One flaky test fails a batch, so the queue bisects or resets to locate the cause, and every speculative state behind the point of failure gets invalidated, which consumes multiple full CI runs in the process of locating a fault that may not even exist. At the arrival rates agents produce, this cascade can consume more CI capacity than the actual development workload it was meant to validate. Agent retry behavior makes the arithmetic worse again: an agent that re-queues a failed PR automatically re-triggers the same cascade for every PR behind it, turning what started as a single flaky test into a delay that touches multiple unrelated changes.

The fix has to move detection upstream, ahead of the merge-blocking suite rather than downstream of a failed merge. Tests that have failed non-deterministically in their recent history should be quarantined from the merge-blocking suite automatically, the moment that pattern is detected, rather than after a human happens to notice the same test failing intermittently for the third time this month. That requires instrumenting test outcomes over time rather than evaluating each run in isolation, a capability most CI configurations simply do not include out of the box.

Validation has to move closer to the agent's own working loop as well. Ability.ai's architecture prescription makes the point directly: if tests take too long to execute, the agent's entire inner loop stalls behind them, and the fix is to make validation run in seconds within the agent's own workflow rather than minutes inside a separate pipeline stage. That requirement, validation fast enough to sit inside the loop rather than outside it, is also a compute problem. Running validation in seconds at the volume agents generate demands an execution layer built for bursty, non-uniform load. The infrastructure underneath the queue has to change next.

Compute scaling patterns that match the non-uniform, bursty load agents create

Agent workloads arrive in bursts, at non-uniform rates, and with sensitivity to cold-start latency, so static or naively auto-scaled CI runner provisioning cannot absorb them. Agents do not follow that rhythm. A single agent working through a multi-step task can produce a burst of PRs in a span of minutes, then go quiet, then burst again, and a runner pool sized for the average load across a day will be badly undersized for the peak and badly oversized for the trough.

This is the same starvation dynamic documented in the SpiceAI incident, but at the infrastructure layer rather than the queue-logic layer. A fixed-size runner pool, however it is provisioned, has no way to reserve capacity for the head of the queue when a burst of lower-priority batches arrives first. Scaling that pool to match agent load means building elasticity around the burst pattern itself, provisioning capacity that can expand within the seconds or low minutes an agent's inner loop demands, rather than the minutes-to-hours window that satisfied a human-paced CI system.

The deeper requirement is a stateful execution environment for agents, one that can preserve context across a burst of related runs without paying the full cold-start cost on every single invocation. Every architectural fix traced through this piece, footprint-aware queues, trust-tiered governance, upstream flake quarantine, converges on the same underlying fact: agents did not break CI/CD by writing bad code. They broke it by moving at a cadence the infrastructure was never built to receive, and the fix at every layer is the same one, building for that cadence rather than tuning around it.