Human Escalation Policies in Agent-Managed Merge Queues

Agent-managed merge queues are buckling under a problem that has nothing to do with code quality and everything to do with arrival rate. The fix is a deliberate policy for where authority over a merge decision sits, and when that authority has to move from an agent back to a human.

An AI agent in a merge queue should stop and escalate to a human when the decision it faces falls in the control plane rather than the data plane, meaning it would change where organizational risk actually sits rather than simply execute within already-authorized boundaries. Specific conditions that cross that line include changes to file paths or API surfaces flagged by tiered trust gates, confidence scores that fall outside calibrated thresholds validated against observed human override rates, and any write action caught by a write-staging policy before the merge executes. The arbitration gate evaluates a tuple of user role, tool name, resource path, environment, and operation scope, and returns an ESCALATE outcome when those inputs exceed the authority the current policy tier allows. Escalation triggered outside those defined conditions is over-escalation, a discipline failure in policy design, not appropriate caution.

Why the merge queue became the bottleneck when agents started writing code

For most of the industry's history, the merge queue was never the slow part of shipping software. A developer spent four or five days writing a feature, opened a pull request, waited on review, and merged. The queue existed to sequence an occasional, human-paced trickle of changes, sized for how fast people typed and thought, not for how fast a CI pipeline could run. Integration was the easy part because production of code was the hard part.

Coding agents inverted that relationship. Qovery's 2026 analysis of the shift states that code now ships many times faster than it used to, while the CI/CD systems, environment provisioning, and deployment pipelines built to receive it were designed around human-speed development. The bottleneck didn't disappear. It moved downstream, into the exact infrastructure nobody rebuilt for the new arrival rate.

The math behind that move is straightforward. Agents don't share that constraint. Multiplying the number of agents working per developer can push the arrival rate past the queue's daily ceiling quickly. At that point wait times stop growing in proportion to the overload and start growing nonlinearly. A queue running comfortably within capacity behaves very differently from one pushed past it, and agents are what pushes teams across that line.

Spotify's experience makes the inversion concrete. Every structural choice downstream of code generation, from CI parallelism to reviewer assignment to deployment gating, assumed a throughput ceiling that agents blow past without even trying. That's the condition every later argument in this piece responds to: once arrival rate exceeds service rate, something has to give, and the only real design choice is whether it gives in a controlled way or an uncontrolled one.

How agent behavior specifically degrades queue performance in ways human behavior did not

Once arrival rate climbs, the failure modes inside the queue change in kind, not just in frequency. Flaky tests are the clearest example. A merge queue typically works by testing proposed changes against a speculative merge state, batching multiple PRs together on the assumption that they'll all pass. One flaky test that fails intermittently invalidates that entire speculative state, forcing every PR behind it to restart from the failure point, regardless of whether those PRs had anything to do with the flaky test. The flake rate of the test suite itself sets the frequency of these cascades.

Uber's experience before it built dedicated flake management illustrates how little slack a system has. Uber's mainline branch was green only about half the time, and the company eventually traced the problem to roughly a thousand flaky tests out of several hundred thousand in its suite. A fraction measured in the low single digits was sufficient to keep the entire pipeline unreliable for every team depending on it.

Agents interact with that fragility in a way human developers rarely did. An agent instructed simply to "get the PR merged" has no equivalent hesitation: it re-queues the same change, burns a full CI run, and in doing so re-triggers the cascade for every PR sitting behind it in line. A behavior that was mildly wasteful when a human did it occasionally becomes something closer to a denial-of-service attack when an agent does it at machine frequency, with no sense that it's degrading the system for everyone else waiting in the same queue.

The Clinejection incident shows where that same structural weakness turns into a security failure rather than just a throughput failure. The mechanism that let a retry loop flood a queue with wasted CI runs is the same mechanism that let a poisoned input escalate into credential theft: an agent with broad tool access operating on inputs nobody sanitized. Retry loops and Clinejection aren't two unrelated problems. Either way, the agent did what it was told, using the access it had been given, and that is the design failure, not a model shortcoming.

When should an agent stop and escalate to a human instead of merging automatically?

The question every team running agents in a merge queue eventually has to answer is which specific operational decisions should move from a human-controlled pipeline to an agent, under what constraints, and with what recourse if the agent gets it wrong. Researchers Barnes and Ghaleb, working out of the University of Toronto and Trent University, give this a name: authority transfer, defined as the delegation of operational decisions from human-controlled pipelines to agent systems under specified constraints and recourse mechanisms. That framing matters because it turns escalation policy into something a team designs up front, rather than a safety net stretched underneath a system after something has already gone wrong.

Barnes and Ghaleb's useful cut is between the data plane and the control plane. Almost every production system today confines agents to the data plane, whether or not the team building it would describe its own governance in those terms, because control-plane authority changes where organizational risk actually sits.

Safin and Balta add a second axis to this picture that sharpens it further: agency, meaning what a system is capable of doing, and autonomy, meaning how much of that capability it exercises without a human in the loop. Their argument is that these two dimensions can't be tuned independently. As autonomy rises, and an agent takes more actions without a human checking each one, the opportunities to catch and correct an error shrink. Agency has to be constrained more tightly to compensate. Giving an agent broad capability and broad independence at the same time is the exact combination that produced the Clinejection incident.

GitHub's own governance of Copilot shows this reasoning applied in production. The question that framework hands back to any team building its own merge queue policy is simple to state even if it isn't simple to answer for every case: is this decision one an agent can own outright with logging, or one it can only propose, with a human required to confirm?

The three structural patterns that dominate how teams are drawing the data-plane / control-plane boundary today

Teams solving this problem in production are converging on three recognizable patterns for drawing that boundary, and none of the three is strictly better than the others. Each trades automation depth against oversight cost differently, and the right choice depends on team size, risk tolerance, and the shape of the codebase.

The first pattern is tiered trust gates. The entire design decision in this pattern lives in the gate condition itself: which file paths, which API surfaces, and which categories of change scope are serious enough to trigger escalation. Get the gate conditions wrong and the tiers are cosmetic.

The second pattern is confidence-threshold escalation. The obvious objection is that confidence scores produced by large language models are not well-calibrated, and a model can be confidently wrong in exactly the cases where confidence should have been low. The model's self-reported confidence number shouldn't be the only input: thresholds should be set against the observed override rate, the rate at which humans actually reverse the agent's decision, with the audit log supplying that signal rather than the model's own assertion of certainty.

The third pattern is deployment-environment stratification, where the authority tier is a property of the environment rather than a property of the PR's content. Preview environments auto-deploy straight from a prompt or a PR, with no gate. Qovery's two-path model makes this split explicit in its own platform: a prompt-to-deploy path built for experimentation, and a Git-push-to-deploy path built for production, both policy-gated and audited on the same system rather than living in separate tools. In both cases, the worktree is the authority boundary, made structural instead of merely written down.

Policy-as-Code as the Enforcement Layer for Escalation Rules

None of the three patterns above survive contact with an agent operating at machine speed unless the rules are enforced at runtime rather than recorded in a document somewhere. An agent does whatever its tool access allows, regardless of what a wiki page says about when it's supposed to escalate. The rule has to live in the same system that grants that access.

The general shape of this enforcement is an arbitration step that evaluates a tuple before any action executes: user role, tool name, resource path, environment, and operation scope. That evaluation produces one of three outcomes. ALLOW lets the action execute, with the event logged. The arxiv paper on AI-augmented CI/CD pipelines names Open Policy Agent, Cedar, and Rego as the specific policy-as-code frameworks providing this safety envelope, giving teams a way to bound what a model can do and audit what it actually did.

Elastic's use of Claude inside its CI/CD pipeline for monorepo dependency management shows what this looks like when it's working. What makes that safe is the scope policy enforces around it: the work is confined to automated dependency-update pull requests and build failures from a specific build tool, a boundary that isn't a suggestion to the model but a constraint the pipeline itself enforces.

Safin and Balta's six architectural tactics, checkpoints, escalation, multi-agent delegation, tool provisioning, tool fencing, and write staging, each translate into a policy-as-code rule rather than a norm teams hope agents will respect. Write staging is the one built specifically for merge queues: it holds a write, a merge in this case, until a human confirms it, converting "the agent should ask before merging" from an instruction into a mechanical gate. None of this works without complete audit logs. Every ESCALATE outcome needs to record the exact policy rule that triggered it, the evidence supporting that trigger, and the agent's own reasoning at the time; that record is what later makes threshold calibration and incident review possible.

The standard objection to all of this is that policy-as-code adds maintenance overhead, and pipelines evolve fast enough that policies can go stale. That overhead is smaller than the overhead of reconstructing, after the fact, what an unconstrained agent did and why. The Clinejection incident is what that second cost looks like when it comes due.

Where human judgment is genuinely irreplaceable versus where it is just friction

Poorly calibrated escalation, whether it escalates too often or too vaguely, degrades the experience of everyone working through the queue and erodes confidence in the system doing the escalating. FutureAGI's 2026 evaluation cohort found that 41% of escalations were flagged as unnecessary when ToolSelectionAccuracy was measured across each escalation span. They're rubber-stamping a queue of decisions the system should have made on its own. The review step has become friction wearing the costume of oversight.

The markers of genuine irreplaceability are narrower than most teams' current escalation rules assume. Outside categories like these, a lot of what gets escalated today is escalated out of habit rather than necessity.

The clearest evidence that removing humans from low-value decisions is a net gain, not a risk, comes from on-call incident response across five channels where agents handled the large majority of incidents without human involvement, and the median time to first root-cause analysis fell dramatically as a result. The value in that result didn't come from eliminating human judgment. It came from stopping humans from being routed into decisions where their judgment wasn't adding anything beyond delay.

Safin and Balta's own framing of escalation applies directly here: escalation is a designed handoff point that a system reaches for on purpose, not a fallback it reaches for when it's unsure what else to do. The test for any given escalation rule is whether the handoff to a human improves the quality of the decision being made or simply adds latency to a decision the system was already equipped to make correctly. Over-escalation is a discipline failure in policy design, not evidence of appropriate caution, and treating it as caution is how review queues end up as overloaded as the merge queues they were meant to protect.

Anatomy of a Well-Designed Escalation Policy

A policy that actually holds up starts with a decision taxonomy. Every action an agent can take inside the merge queue gets classified as data-plane or control-plane and assigned a default authority tier. That taxonomy is the document that makes the rest of the system writable as policy-as-code, because nothing downstream can be enforced if the classification itself hasn't been made explicit.

The ALLOW, BLOCK, ESCALATE arbitration pattern routes every classified action as it happens, with ESCALATE serializing the action and handing it to a human along with the full audit context behind the decision, not just a bare notification that something needs attention.

A calibration loop closes the gap between policy and reality. Tracking the human override rate per rule tells a team which rules are working and which aren't: a rule with a near-zero override rate is a candidate for promotion to fully autonomous status, while a rule with a high override rate needs either a tighter threshold or a different classification. FutureAGI's 2026 source puts a number on where that balance should sit, noting that in stacks running models like GPT-5.x or Claude Opus 4.7, the right escalation rate for support workloads sits above 8% but below roughly a fifth of all interactions, with rates outside that band signaling that something in the policy is miscalibrated.

Production signals keep that loop running instead of letting it go stale. The arxiv CI/CD paper's own recommendation follows the same logic at the metrics level: measuring lead time and deployment frequency alone, the standard DORA metrics, misses the part of the story specific to agents. The paper argues for tracking intervention accuracy and human override rate alongside them, as the indicators that actually show whether an authority transfer is working.