Dead Skill Detection in AI Coding Agent Harnesses

Dead skills pile up in AI coding agent harnesses because adding one costs nothing, but proving one is dead takes real engineering time. The result is a prompt that carries capabilities no one has touched in months, taxing every single turn with context the agent will never use.

Why Skills Accumulate, and Why Removal Is Hard

A skill gets declared, described, and loaded into the prompt on every turn, whether or not any task that turn will ever touch it. The agent pays the cost of carrying the skill regardless of whether it fires, and this is the first asymmetry worth understanding, visible even before registries compound the problem on the way in. Screening is largely automated and post-publication, and while npm now offers a publish-time human review gate, no registry mandates human pre-publication review across the board, which keeps onboarding friction low. Low friction in means skill libraries grow organically, and growth without a matching discipline on the way out means libraries only ever expand. Nobody prunes symmetrically to how they add, because adding a skill requires no justification while removing one requires proof that nothing depends on it. That proof demands telemetry most harnesses simply don't collect by default: a record of which skill fired on which run, matched against which runs needed it and which didn't. Skills overlap with each other, reference conventions that have since changed, or quietly drift away from how the capability actually gets used in practice, and none of that drift trips any alarm on its own. The gap between what a skill claims to do and what it is actually needed for stays invisible until someone goes looking, and going looking is exactly the expensive step nobody has budgeted for. That's the telemetry gap the rest of this piece works through: not a tooling inconvenience, but the structural reason dead skills survive as long as they do.

What "Dead" Actually Means, and Why the Categories Matter for Pruning Decisions

A skill can be dead in at least three distinct ways, and treating those three as one condition leads to the wrong pruning decision every time. The skill-decay tool at Skills.lc draws the lines with a three-tier taxonomy: never (declared, loaded, and invoked zero times across the logs), stale (used once but not within the last N days, with 30 as the default window), and live (the skill is actually earning its spot in the prompt). A "never" skill isn't automatically safe to delete. A break-glass capability, something built for disaster recovery or a rare emergency path, should sit at zero invocations for long stretches by design, so zero is a signal to investigate, not a verdict to act on.

Stale is the more dangerous category in practice, because a stale skill was used at some point, meaning its instructions may still be actively followed by an agent even though the system it describes has moved on. A staleness survey of a developer skills library, run on 2026-09-23, found tool facts in seven skills directly contradicted by current versions: a PDF backend built on PyPDF2 that no longer reflects the library in use, a Toss coverage gate that ignores a newly documented WebSocket API, CLI version references that had gone out of date, a repository link returning a 404, and display names and URLs that had simply changed underneath the skill's own description. All seven were actively wrong, which is worse than unused, because an agent following them produces confidently incorrect behavior rather than simply skipping a step.

A third failure mode sits beneath both of these, produced by a skill that is genuinely invoked but whose telemetry column was never filled in the first place, leaving the monitoring system built to catch dead skills with no data to work from. That's the exact shape of the oxagen/macanderson case covered below. A "never" skill warrants an investigation into intent, asking whether it's a rarely-needed safeguard or genuine dead weight. A "stale" skill warrants a content audit against whatever upstream truth it's supposed to reflect. And a skill with broken or missing telemetry warrants fixing the instrumentation first, because no pruning call is trustworthy until the data behind it is.

What dead skills actually cost per turn, and why the cost compounds

The cost of a dead skill isn't measured only in tokens. Latency rises as context grows, and a model's attention is a limited resource that gets split across every capability declared in the prompt, whether or not that capability is relevant to the task at hand. In Vercel's evaluation suite, the Next.js docs skill went uninvoked by the agent in 56% of test cases. Its context overhead was paid in full on more than half of all runs for a capability those runs never touched. Every one of those runs spent latency and attention on a skill that did nothing for it.

That per-turn tax doesn't stay flat. As AI-generated code accelerates how fast pull requests get produced, agents run through more turns in the same stretch of time, and a tax that was easy to absorb at human working speed starts to add up at agent speed. A cost that was a rounding error when a handful of developers triggered a handful of agent runs a day becomes a running expense once agents are firing continuously across parallel tasks.

The right response to a rising per-turn cost is a CI gate: fail the build once the count of never-used or stale capabilities crosses a budget set in advance, so the team has to prune the skill or explicitly justify keeping it at merge time, rather than waiting for a quarterly audit to notice. A quarterly audit catches the damage after months of wasted turns. A build gate catches it before the next merge.

The oxagen/macanderson case: when dead-skill telemetry is itself dead

The most dangerous version of a dead skill is one whose telemetry was never recorded at all, because it means the system meant to catch dead skills can't be trusted to catch anything. The oxagen/macanderson codebase offers a documented instance of exactly this. ADR-043, dated 2026-09-07, retired the skill system there in its entirety: packages/skills, the skill. and agent.skill. contract families along with their API routes, MCP tools, handlers, and app pages, and the agent.skills and agent.skill_versions database tables that backed them.

The audit that led to that decision found a specific and telling pattern. The module had no production caller left once agent.skill.load had been deleted; the only code still touching it was its own test suite, including a reference at barrel.test.ts lines 61 through 62. Nothing in the system recorded which run had used which skill. The execution_step_id column on skill_loads, the field that should have linked a skill invocation back to the run that triggered it, had never been populated. A monitoring system existed in name, and it had no data behind it from the start.

The downstream effect reached the product surface. The legacy Skills page in the application couldn't be migrated or ported forward, because the functionality it was supposed to display had already stopped existing behind it. The only way the team got to a clean answer was a manual audit of callers, tracing by hand who still touched the module rather than trusting a dashboard that had never been wired up to tell them. The lesson generalizes past this one codebase: instrumenting the skill-load event has to happen before any detection discipline means anything, because a system can't surface the death of something it never recorded.

How the britt/agent-skills audit established repeatable deprecation criteria

A structural cleanup of a skill library is not the same exercise as a content audit, and mistaking one for the other leaves dead instructions sitting untouched. The britt/agent-skills review, conducted in July 2026, found exactly that gap. A prior pass through the library had fixed frontmatter formatting, repaired broken cross-references, and bundled related skills together, but it had never asked whether each skill's actual instructions still matched reality or whether the capability was still needed at all.

The second pass, built specifically around content, set out deprecation criteria in advance rather than deciding case by case: a skill gets marked for deprecation when its functionality has been fully superseded by another skill or by a bundle, or when it depends on a system or integration no longer supported. Those criteria, applied across the library, produced a concrete result: one skill, consolidate-notes-summary, was deprecated outright; six were updated to bring their instructions back in line with current reality; and four were bundled together to cut down on overlap.

Structural health, meaning clean frontmatter, working links, accurate metadata, is a different question from content health, meaning whether the instructions are still accurate and whether anyone still needs them. Each needs its own pass, with its own checklist, because a skill can pass every structural check and still tell an agent to use a tool that no longer exists. And the criteria for deprecation need to be written down before the audit starts. Judgment calls made in the moment, skill by skill, produce inconsistent outcomes and arguments about edge cases that a standing rule would have settled in advance.

Detecting dead skills systematically: from usage logs to staleness checks to CI gates

Detecting dead skills works as a three-layer discipline, moving from usage-log triage to content staleness verification to enforced budget gates, where each layer catches what the one before it misses.

The first layer is usage-log triage. The skill-decay tool at Skills.lc runs the never/stale/live taxonomy directly against invocation logs, and one run against a 42-skill inventory turned up 7 skills in the "never" category and 5 flagged as "stale". This layer only works if the skill-load event is instrumented in the first place; the oxagen/macanderson case is what happens when that prerequisite is skipped. The log window matters as much as the instrumentation. A window set too short makes a skill that's used monthly look abandoned, so teams need to declare their window explicitly and set it to match their own release cadence rather than borrowing a default that fits someone else's schedule.

The second layer is content staleness verification, applied to whatever skills survive layer one with a "live" label. This means checking each surviving skill against upstream truth: current API documentation, repository links that still resolve, library versions still actually in use. The 2026-09-23 staleness survey is the clearest model for what this looks like in practice, turning up seven skills whose stated facts had been contradicted by current versions, each needing a targeted correction rather than a deletion. This layer only works if someone maintains a reference to the canonical upstream source each skill is supposed to reflect; without that anchor, "staleness" has nothing to be measured against.

The third layer is the CI budget gate, which turns detection into enforcement. A team sets the maximum acceptable count of never-used or stale skills as a build constraint, and CI fails once that budget is exceeded, forcing a prune-or-justify decision right at merge time instead of at the next scheduled audit. The gate has to distinguish between categories to be useful: a "never" skill can be justified as a break-glass capability kept deliberately idle, but a "stale" skill carrying wrong API facts has no comparable justification.

The skillsaw linter's ecosystem assessment, dated 2026-10-01, surveyed competing linters and ranked feature priorities, and it stopped short of claiming that dead-skill detection tooling is already distributable through the Vercel skills CLI channel it was auditing, which points toward the detection layer eventually becoming installable as a skill in its own right. That installation path isn't universal across harnesses, either. A parallel wave of agent harnesses, each tied to its own open-weight model release, its own CLI, and its own skill contract, means a skill written for one may be structurally unreachable in another.

Dead skills as a factory-floor input problem, not a cleanup task

Dead and stale skills aren't just a tidiness issue sitting downstream of the real engineering work. They contaminate the input to every agent turn, and that contamination doesn't stay contained to the turn where it happens; it cascades straight into the merge queue. When an agent runs on stale skill instructions, it produces code that fails CI for reasons the agent itself cannot diagnose correctly, because the instructions it was given describe APIs, file paths, and tools that no longer exist. The agent isn't wrong about reasoning. It's wrong about the facts it was handed.

The merge queue is where this cost lands hardest, because the merge queue has become the actual bottleneck in agent-driven development. One developer directing several agents across parallel worktrees can produce pull requests at a rate that no longer tracks headcount at all, but merge throughput still tracks the same constraints it always has: test suite duration, runner capacity, flake rate, and the mechanics of the queue itself. None of those got faster just because the agents got faster.

An agent working from a stale skill that's simply instructed to "get the PR merged" will re-queue the same flawed change over and over, burning a full CI run on every attempt and triggering cascade failures for every other change waiting behind it in line. A retry loop that was mildly wasteful at the pace a human reviewer works becomes something closer to a denial-of-service attack on the queue once it's running at agent frequency.

When agents running on stale skills submit flawed pull requests into that environment, they make both of those numbers worse. Seen this way, dead-skill detection functions as factory-floor sanitation: it keeps the raw material feeding each agent clean, so the queue downstream doesn't have to absorb the cost of bad instructions repeated at scale. An agentic merge queue, one that manages CI failures, conflict resolution, and its own state without a human in the loop at every step, is only as reliable as the skills instructing the agents that feed it. Skill hygiene and queue mechanics are the same discipline, applied at two different points in the same pipeline.

Why version-controlled skill

libraries matter follows directly from everything above: every layer of detection described here, usage-log triage, staleness verification against upstream truth, and CI budget gates, depends on being able to see a skill's history, not just its current state. A skill's frontmatter, its instructions, and its deprecation status all need to live under the same version control discipline applied to the rest of a codebase, so that a staleness check has something concrete to diff against and a CI gate has a clear point in history to enforce its budget at. Version control doesn't just record what a skill says today. It's the mechanism that makes tomorrow's audit cheaper than today's.