SkillOpt: optimising markdown skill files as trainable parameters
Muratcan Koylan unpacks SkillOpt (arXiv 2605.23904), one of the first papers to treat markdown skill files as trainable parameters with a real optimization framework. His read: the validation gate is the only thing that matters in a self-editing loop — held-out set, strict improvement, ties rejected; the best skills converge in just 1-4 accepted edits, and an agent that accepts most of its own proposals is shipping slop. Bounded edits (4-8 per step) beat full rewrites — the textual analog of a learning rate — compactness wins (median final skill ~920 tokens), and the skill now matters more than the harness: a Codex-trained skill ported into Claude Code gained +59.7 points on SpreadsheetBench, because procedural knowledge generalises beyond the runtime that produced it. The catch he flags: verification is the bottleneck — every gate leans on an auto-grader, which holds up for benchmarks but breaks down on open-ended writing, design and strategy.
Was this useful?