The prototype rendered perfectly. Clean layout, working interactions, valid markup — everything I would have thought to check for. It also looked nothing like our product, because the skill had ignored our internal design system completely and wildcatted its own CSS from scratch.

That was the first few rounds of an internal UI prototyping skill I still maintain, and it’s a worse failure than a screenshot makes it look. The skill exists to produce prototypes that do two things at once: read as ours against the internal design system, and fold into an existing application without setting off a UI rewrite and a styles audit. Hand-rolled CSS misses both. It looks wrong, and adopting it means redoing the styling by hand — which is the work the prototype was supposed to save.

So it failed. Quietly, though. Nothing was broken, every check I would have written passed, and the output was only wrong in a way you can see when you hold it against the thing it’s supposed to match — which is to say, on a real project, not in a test.

Finishing an internal skill — one of the reusable AI workflows I build for things I do repeatedly: UI prototypes, cleaning datasets, standing up quick data displays for stakeholders — means running it on real work, over and over, and reading back what I had to do to compensate. It takes three to five rounds before I’ll use one without babysitting it. But the iteration count isn’t the signal I actually watch. It’s whether the last two rounds produced more documentation than code changes.

Here’s what the loop looks like.

What I Look at Between Rounds

The raw material for each iteration comes from three places: the artifacts the skill produced, the session transcripts from the runs themselves, and the edge cases real work throws at it.

The artifacts tell me whether the output was right. The transcripts tell me why it went wrong — what questions I had while running it, what I had to re-prompt, where the instructions were ambiguous enough that I compensated with extra context. Those re-prompts are the most valuable signal in the loop. They’re the skill’s debt, written in plain English.

Finding the transcripts

If you’re running Claude Code, every session is already being recorded and you didn’t have to turn anything on. Each session is a JSONL file — one line per event, covering your prompts, the model’s replies, and every tool call and result.

They live here:

~/.claude/projects/<dashed-project-path>/<session-id>.jsonl

The directory name is your project’s absolute path with the slashes swapped for dashes, so /Users/nathan/Projects/blog becomes -Users-nathan-Projects-blog. One file per session, named by session ID.

To read one as a conversation, /resume inside Claude Code lists recent sessions and reopens whichever you pick, and claude -c drops you back into the most recent one.

For iterating on a skill, though, I don’t want the conversation. I want my half of it. This pulls every prompt I typed, in order:

jq -r 'select(.type=="last-prompt") | .lastPrompt' session.jsonl | awk '!seen[$0]++'

The dedup is doing real work — that record is a running checkpoint, so a prompt repeats once for every turn it stayed current. Long prompts come back truncated, which is fine for scanning.

What you get is the session’s skeleton:

I want to come up with a flow that allows me to use the hermes agent gateway…
the flow should be that the hermes agent (through maybe a skill) creates an issue in github…
it should be a hermes skill not a claude one
I had restarted it before sending the emial

Line three is the whole point. I had to say it, which means the skill hadn’t. Every correction like that is a missing instruction or an ambiguous one, and reading twenty of them in a row tells me more about what to fix than re-reading the skill file does.

One caveat: this file layout is internal, not a documented API. It has changed before and will again, so check the record types against your own transcripts before you build anything on top of them.

Edge cases

Edge cases hit in practice are the third input.

The same prototyping skill missed a second time, later and for a different reason. I export UI prototypes for AI products as a single-page HTML build through Vite — it’s how I hand someone a prototype without asking them to install anything first. The skill didn’t do that. It produced a perfectly reasonable multi-file project instead, a good answer to a question I hadn’t asked.

That one wasn’t a bug. The skill’s own checklist never said the export pattern out loud, because to me it isn’t a preference; it’s just how prototypes work. Every real project surfaces at least one of these, and how the skill handles them is usually what separates a v1 from something durable.

Layering in Evals

After a few rounds, I add more formal checks.

Deterministic checks. Does the output have the right structure? Are required fields present? Does the file end up where it’s supposed to? Cheap to write, fast to run, and they catch the most embarrassing failures before I ever see them.

LLM-as-judge criteria. A short rubric — four or five criteria — that a model scores the output against. The criteria worth writing are the ones I derived from failure modes I’d already hit, not abstract dimensions like “clarity” or “completeness.” “Does every color and spacing value come from a design system token instead of a raw hex or pixel value?” tells me something. “Is the styling good?” doesn’t.

A clean-room reviewer that loops. Scoring the design system problem turned out not to be enough, because a score just tells me the run was bad after I’ve already got the output. So the skill now hands its generated styles to a subagent in a fresh context — one that never watched the prototype get built and has no stake in any of the choices, so it can’t rationalize a hand-rolled value as deliberate. Its job is narrow: flag anything that isn’t a design system token. What it flags goes back for another pass, and the loop runs until the review comes back clean.

That’s still a model judging output. The difference is that it acts on the verdict instead of reporting it, and it’s the only one of the three tiers that has actually fixed the problem this post opened with.

This adds real overhead. It also changes what I’m willing to trust: a skill whose evals pass feels different from one that merely seems to work.

The Bar for Sharing

The jump from “good enough for me” to “good enough for a teammate” is bigger than I expected the first time I made it.

What I consistently underestimate is how much implicit context I bring to running my own skills. The skill isn’t just the prompt — it’s the prompt plus everything I know about when to use it, how to prepare the inputs, and what “good” looks like. I know which inputs are clean and which need pre-processing. I know the failure modes because I hit them. A colleague opening it for the first time has none of that, and they walk straight into the assumptions I forgot I was making.

So sharing means another round: explanation and guardrails that felt unnecessary when it was just me.

Read the output as someone else

The check I always run last and should run first is on the wording the skill produces. A skill isn’t done when the output is correct. It’s done when the output is correct and the person who asked for it can read it.

Models have house dialects. Fable has its own tics, and every model drifts toward technical register when the task is technical. Left alone, a skill I wrote for myself will hand a product manager a paragraph about token budgets and schema validation and consider the job finished. It’s accurate. It’s also useless to the person who asked.

So before I share anything, I read a sample output the way a non-expert would — usually a technical PM, since that’s who most of my skills eventually serve. Where would they stop? Which word would they have to look up, or worse, guess at and get wrong? Then I write the answer into the skill: name the audience, ban the specific jargon I keep seeing come back, and include one example of the register I want.

That’s usually the biggest gap between the version I use and the version I hand over. The logic barely moves; the wording gets rewritten end to end.

Two Places a Skill Can Live

Even once a skill clears that bar, it doesn’t go straight into my organization’s skill marketplace. It gets a standalone GitHub repo of its own first.

That repo is deliberately temporary, and the boundary it draws does more for me than any amount of documentation. A skill sitting in a repo of its own announces what it is: a work in progress, probably fragile, actively looking for someone to break it. I can send a colleague the link and say “try this, it’ll fail on something” without that being an odd thing to say. The container already said it.

Recruiting beta testers gets much easier from there. Trying an experiment is a small ask. Trying something that lives in the marketplace carries an implied promise that it works, and when it doesn’t, I’ve spent trust I’d rather save for the version that does.

The marketplace is the stable channel. Getting into it means I’ll support the skill and keep updating it as new edge cases come in, so I’m deliberate about what graduates.

By the time something graduates to the marketplace, it’s been through several more rounds than the original three to five. Even then, the first few people to run it in the wild surface something I missed.

How I Know It’s Done

Three to five iterations is a heuristic, not a rule. Some skills are simpler than they look and stabilize fast. Some look simple and take twice as long.

The number worth watching is the ratio. When the last two rounds produce mostly new edge cases in the docs and barely touch the core logic, the skill has stopped changing and started being explained — that’s usually close to stable. When rounds are still rewriting logic, it isn’t done, however many I’ve run.

If you’re building skills for anyone but yourself, that’s the ratio to track from round one.