Train the skill, not the weights.

The big picture, in plain English

Before any equations: here is the whole paper in three pictures. If you read nothing else, read this.

Modern AI agents follow a skill — a page of natural-language instructions telling them how to do a task: which tools to call, what format to output, what mistakes to avoid. Today those pages are hand-written, generated once and frozen, or rewritten by the model in loosely-controlled ways. None of them reliably get better the way a neural network gets better during training. SkillOpt asks: what if we optimized the skill page the same disciplined way we optimize model weights?

What they did, and why it’s surprising

Frontier agents adapt to a domain through their procedures — how they gather evidence, call tools, format answers. SkillOpt makes that procedure layer trainable, without touching a single model weight.

Given a domain, an initial skill, and a frozen target model, SkillOpt loops: it samples batches of scored rollouts, hands the successes and failures to a separate optimizer model, and asks for bounded add / delete / replace edits to the skill. It ranks and clips those edits under a textual learning rate, applies the bounded update, and then — the crucial step — evaluates the candidate skill on a held-out selection split. An edit survives only if it strictly improves that score.

The deployed result is a compact best_skill.md (≈300–2,000 tokens) built from just 1–4 accepted edits. It adds zero optimizer calls at deployment — the agent simply reads a better page. Across six benchmarks, seven target models, and three execution harnesses, SkillOpt is best or tied-best on all 52 evaluated cells.

If skills are the adaptation layer, how should they be optimized?

Frontier models are increasingly deployed as agents — from single-prompt callers to multi-step harnesses with tools, files, and verifiers. In those settings, adapting to a domain isn’t only about weights or prompts: it’s about improving the procedures the agent uses. Agent skills are the natural interface for that — a portable text artifact packaging procedures, heuristics, tool policies, output constraints, and failure modes.

But weight adaptation is unavailable for closed frontier models and expensive for open ones, while hand-written or one-shot skills are brittle. Recent systems convert experience into textual artifacts — distilling trajectory lessons, refining skill folders, building skill libraries — yet leave a more basic question open:

The key move: treat skill editing as a controllable domain-adaptation process. The skill document is the external state; an additional frontier model is the optimizer; and training-style controls govern evidence, step size, validation, and update direction. The stakes are practical — if consecutive revisions move too far or in inconsistent directions, the optimizer can no longer learn from what helped and what failed. Stability is the whole game.

The deep-learning analogy is operational, not decorative

Every design choice maps onto a piece of the standard training loop. The skill document plays the role of parameters; trajectory feedback plays the role of the gradient; and a suite of familiar controls keeps the process stable.

Two roles stay strictly separated, and this separation is what makes the method cheap to deploy:

One optimization step, end to end

SkillOpt is a loop. Each step is a forward pass (gather evidence), a backward pass (reflect into edits), a bounded update (apply within budget), and a gate (accept only if validation improves). Across epochs, a slow/meta update carries longer-horizon lessons. Click a stage to expand.

Problem setup: what is being optimized

A skill s is a natural-language policy inserted into the agent’s context before execution. For a harness h, task x, and skill s, running the frozen model M produces a trajectory and a scalar score:

The data is split three ways. The train split D_tr supplies rollout evidence; the selection split D_sel gates updates; the test split D_test is locked until the final report. SkillOpt generates candidate skills, picks the best on selection, and reports on test:

Forward pass — rollout evidence

At each step the target model runs a rollout batch from D_tr with the current skill. The harness records everything the optimizer might learn from: messages, tool calls, observations, command outputs, final answers, verifier feedback, and benchmark-specific context (spreadsheet previews, document references, compact execution traces). This batch is the evidence unit: small batches update quickly but noisily; larger batches expose recurring patterns before the skill changes.

Backward pass — minibatch reflection

The optimizer turns trajectories into edits. It first separates failures from successes, then partitions each group into reflection minibatches. This matters: single trajectories produce anecdotal fixes, but a minibatch reveals reusable procedural errors — the agent consistently searches the wrong source, formats the answer wrong, or never verifies a tool result. Failure minibatches propose corrective rules; success minibatches preserve what already works. Proposals are then merged hierarchically — failure and success edits consolidated separately, then combined with priority on failure corrections — filtering duplicate, contradictory, and example-specific suggestions.

The bounded update — a textual learning rate

This is the heart of the method. After aggregation, the optimizer ranks the merged edit pool by expected utility and clips it to the top L_t edits. That budget L_t is the learning-rate analogue. Unbounded rewrites can erase useful rules, introduce contradictions, or overfit to a single failure; a bounded update preserves continuity while still letting the skill acquire new procedures. The budget follows a schedule — constant, linear, cosine, or autonomous — with the default cosine starting larger and decaying toward small consolidation steps.

The validation gate & rejected-edit buffer

Every candidate skill is evaluated on D_sel with the same frozen model and harness. If it improves over the current selection score, it becomes the new current skill; if it also exceeds the best so far, it becomes best_skill.md. Otherwise it’s rejected. The gate is intentionally strict — strictly-greater, ties rejected — so the deployed skill never silently drifts. This turns reflection into propose-and-test optimization rather than unconditional self-editing, which is crucial because plausible textual diagnoses can still hurt the actual target model.

Rejected updates aren’t wasted. An epoch-local buffer records observed failure patterns and, for each rejected step, the edits tried and the score drop they caused. Later reflection calls in the same epoch receive this buffer, so the optimizer avoids repeating failed edits and focuses on unresolved failures — negative feedback during training, with no inference-time cost.

Epoch-wise slow / meta update — the momentum term

Fast updates learn from the current batch; the slow/meta update learns across adjacent epochs. At an epoch boundary, SkillOpt re-runs the same training items under the previous-epoch skill and the current skill, then sorts the results into improvements, regressions, persistent failures, and stable successes. The optimizer writes a concise longitudinal guidance block into a protected slow-update field that step-level edits cannot overwrite — and even this candidate passes the validation gate. A separate meta skill lives only on the optimizer side: it summarizes which edit patterns helped, which were rejected, and which failures persist, and is prepended to future optimizer prompts — but it is never shipped with the deployed skill.

The full procedure (Algorithm 1)

The state variables are the current skill s_cur, the best validation-gated skill s_best, a selection-score cache (so identical candidate skills aren’t re-evaluated), the rejected-step buffer, and the optimizer-side meta skill. In sequence, per step: collect rollouts → split into failure/success minibatches → ask the optimizer to analyze each → merge (failure, then success, then failure-prioritized final) → rank and keep at most L_t → apply → gate on D_sel → accept or push to the rejected buffer. At each epoch boundary (from epoch 2): the slow update and meta-skill update run. Only s_best is exported.

Best or tied on all 52 cells

The main experiment is a matrix: seven target models × six benchmarks in direct chat, plus GPT–5.5 under the Codex and Claude Code harnesses. The benchmarks span single-round QA (SearchQA), spreadsheet code execution (SpreadsheetBench), multi-turn document tool loops (OfficeQA), multimodal document QA (DocVQA), mathematical MCQ (LiveMath), and embodied decision-making (ALFWorld). Baselines: no skill, human skill, one-shot LLM skill, Trace2Skill, TextGrad, GEPA, and (in harnesses) EvoSkill.

The same loop works inside agentic harnesses

SkillOpt isn’t a direct-chat trick. The identical optimizer drives GPT–5.5 inside the Codex and Claude Code CLIs, where the skill becomes persistent procedural memory and the harness reads back a compact execution trace for the optimizer to learn from.

The full matrix

The complete Table 1, all models and baselines. Within each model–harness block, the best measured entry per benchmark is in green; SkillOpt rows are highlighted.

Direct-chat block: six benchmarks, seven models. Harness blocks: GPT–5.5 only, five benchmarks (ALFWorld requires persistent embodied interaction not represented in standard Codex/Claude Code adapters). All scores are held-out test percentages.

Which controls actually matter?

Using GPT–5.5 as both target and optimizer, the paper removes one mechanism at a time. The headline: gains are insensitive to the exact batch sizes and schedule, but highly sensitive to the presence of bounded text-space learning, validation gating, rejected-edit feedback, and the slow/meta update — the choices that make skill editing behave like a controlled training loop.

Evidence beats fragility

How much training evidence does the optimizer need? Procedural benchmarks reward more — SpreadsheetBench climbs steadily as the optimizer sees more of the training partition — while QA saturates early. The point is that the headline gains come from having enough scored evidence per update, not from a fragile prompt-search batch size.

The other knobs are reassuringly flat. Sweeping the reflection minibatch from 1→32 keeps SearchQA inside 85.9–87.1; sweeping the rollout batch from 8→full-epoch keeps it inside 85.1–87.2. And on the learning rate itself, every moderate bounded budget (L_t∈{1,2,4,8,16}) beats the unbounded-rewrite baseline — the qualitative claim (bound your edits) holds regardless of the exact scheduler.

The skill is an artifact, not just a prompt

If the learned skill encodes real procedural knowledge, it should survive being moved. The paper tests three shifts — across model scales, across execution harnesses, and across nearby benchmarks — and every measured transfer is positive (no row falls below the target’s no-skill baseline).

Optimizer strength is a free training-time lever

Because the optimizer never runs at deployment, a stronger optimizer buys larger gains at zero inference cost. A strong frontier optimizer (GPT–5.5) beats a target-matched one on every cell — but the target-matched optimizer still recovers 56–74% of the gain. So SkillOpt is not just distilling a strong teacher into a weak student; the bounded, gated loop itself contributes most of the value. (And the gate is what keeps a stronger optimizer monotone — without it, a stronger model could just as easily push larger but harmful rewrites.)

Compact, cheap, and procedural

A central premise is that the trainable object stays a small, inspectable text document. It does. Final skills range from 379 tokens (LiveMath) to 1,995 (SpreadsheetBench), median ≈920 — well under a typical system-prompt budget. And the gains come from astonishingly few accepted edits: LiveMath’s +29.3-point gain and OfficeQA’s +39.0 each arise from a single accepted edit. The optimizer proposes many more per epoch; only a handful pass the gate.

From Table 6 (GPT–5.5 student / GPT–5.5 teacher). Cost/pt is training tokens per absolute test-point gain. Two regimes: cheap procedural benchmarks (0.6–3.6M tokens/pt) and expensive long-trajectory or multimodal ones (SearchQA 37.9M, DocVQA 46.4M). All of it is paid once during training — deployment adds zero optimizer calls.

One real rule per benchmark

Verbatim from the final best_skill.md of each run (Figure 4). Notice: every rule is procedural, not instance-specific — none names a particular question, file, or entity — and each encodes discipline frontier models lack zero-shot.

Two qualitative case studies confirm the shape. The ALFWorld skill evolves from a generic search-transform-place plan into a finite-state execution policy with object identity, search memory, and loop breakers (49.3→74.6 in that run). The SpreadsheetBench skill turns generic Python automation into a workbook-forensics policy — inspect the real workbook, normalize cell types, write static values (40.4→78.9).

Limitations & a skeptic’s note

• Needs reliable feedback. The loop relies on scored trajectories and a held-out gate, so it’s most directly applicable when the task has automatic verifiers, exact-match metrics, or executable checks. Open-ended, subjective domains would need stronger human or model-based evaluation.
• Training isn’t free. Deployment is cheap, but training spends rollout compute and optimizer calls — order 0.6M–46M tokens per test point. Amortized over reuse, but less attractive for one-off tasks.
• One skill, one domain. By design SkillOpt optimizes a single portable skill rather than a library — simpler to deploy, but possibly insufficient for highly heterogeneous domains needing many disjoint procedures.
• Distribution-bound heuristics. Optimized skills can encode training-distribution heuristics, so held-out evaluation remains necessary before transferring to substantially different settings.

What to remember

• The skill is the trainable state. Treat the skill document as parameters of a frozen agent and optimize it like weights — the analogy is operational, every control earns its place.
• Bound the step. A textual learning rate (the edit budget) keeps revisions close enough that the optimizer can learn from its own history. Unbounded rewriting loses to it.
• Gate everything. Accept an edit only if a strict held-out check improves — propose-and-test, never unconditional self-editing. This is what prevents plausible-but-harmful edits from accumulating.
• Failures are feedback. The rejected-edit buffer turns dead ends into negative training signal at zero deployment cost.
• Momentum across epochs. A protected slow/meta update carries durable lessons; removing it is among the most damaging ablations.
• It transfers. The exported best_skill.md retains value across model scales, harnesses, and nearby benchmarks — optimize once, audit as text, reuse.

The broader bet: by making the skill itself the trainable object, the full toolkit of optimization — learning rates, schedules, regularization, curricula, validation — becomes available to a part of the agent stack that has so far been hand-engineered.