2026-09-04
The skill that never fires
Docs packed into a skill still wait to be asked. Vercel Next.js 16 evals: the agent never invoked the skill in 56% of runs — the same knowledge in AGENTS.md did not wait.
On-demand retrieval is a decision. Agents skip it. Pack the docs into a skill and you have not shipped the docs — you have shipped a toolbox entry the model may never open. Vercel Next.js 16 eval suite made that measurable: default skill approximates no-docs baseline around 53%; use-the-skill wording pushed invocation to about 79%, and that wording is brittle.
A compressed ~8KB docs index in AGENTS.md hit 100% on that suite. One eval is not a law, but the direction is clear: always-on context beats a toolbox the agent has to remember to open. Skills still win for vertical, explicit actions — run this migration, call this MCP, clean up this flag — where the trigger is obvious.
The cousin failure is a stale always-on file. aliou.me AGENTS.md still teaching Kysely after the project moved to Drizzle is worse than no file: the agent follows yesterday stack with confidence. Prefer a small live index over a large stale one, and keep skills for the jobs that actually need a named tool.
Subscribe: /log/feed.xml