Español

Audit Your Skill Library Quarterly With Grep

Your AI instructions are already out of date.

Skills rot like code, and they rot silently. Files drift, unreferenced skills pile up, names change, and tool references go stale — but nothing errors out. Your agent just keeps confidently loading the wrong thing. Code has a test suite to catch decay; your skill library usually has nothing. So put it on a calendar: audit every quarter with grep, and treat the audit as a review pass, not a delete pass.

Why skills rot silently

There's no compiler for prose. A skill can be wrong for months and never throw:

• Unreferenced files accumulate. A skill gets superseded but never removed. It still loads, still influences the agent, still costs context.
• Front-matter drifts. A skill's name, description, or trigger conditions stop matching what it actually does, so it fires at the wrong times.
• Tool names go stale. Guidance still mentions tools you retired — tslint (replaced by ESLint), bower (dead), a renamed internal script. The agent follows instructions that no longer map to reality.

None of this shows up as an error. It shows up as an agent that quietly does the wrong thing.

The mechanism: a quarterly grep audit

You don't need special tooling — grep and ripgrep find most of the rot in minutes.

Find unreferenced skills. For each skill, check whether anything else points at it:

# For every skill file, is its name referenced anywhere else?
for f in skills/**/*.md; do
  name=$(basename "$f" .md)
  refs=$(rg -l --glob '!'"$f" "$name" . | wc -l)
  echo "$refs  $name"
done | sort -n   # zero-reference skills bubble to the top

Check front-matter drift. List every skill's declared name and description and eyeball them against what the file now does:

rg -n '^(name|description):' skills/ --glob '*.md'

Hunt stale tool names. Grep for tools you've retired or renamed:

rg -n 'tslint|bower|yarn run|old-internal-cli' skills/

Unreferenced means "review," not "delete"

This is the part people get wrong. A skill with zero inbound references is flagged for review, not condemned. Some unreferenced files are legitimate:

• Deep-dive children loaded on demand through a parent skill via progressive disclosure — the parent points at them, but a naive grep against the wrong pattern misses that link.
• Entry points invoked by the runtime or by the user directly, not by another file.

So the audit produces a list to investigate. For each flagged file you decide: still used through a parent (keep), genuinely dead (delete), or drifted (fix). Never bulk-delete on the grep output alone.

Power moves

• Put it on the calendar. A recurring quarterly reminder beats "when it gets bad," because you can't feel silent rot.
• Keep a retired-tools blocklist. Maintain a grep pattern of tool names you've dropped, and rerun it every audit so new stale references get caught fast.
• Diff descriptions against behavior. If a skill's description no longer predicts when it should fire, fix the front-matter — that's what the agent matches on.
• Trace the parent links. Before deleting an unreferenced deep-dive, grep for it by path and by filename to catch progressive-disclosure references.
• Audit after big renames. A directory move or tool migration is the moment stale references are born; run the audit then, not just on the quarter.

Resources

• ripgrep — user guide
• Agent Skills — Anthropic docs (progressive disclosure)
• grep — GNU manual
• ESLint — the successor to TSLint (deprecated)