Claude Code plugin eval: gate skills on delta before you merge

UpdatedSeptember 14, 2026

Claude Code now ships a first-party way to prove a plugin or skill actually helps. The command is claude plugin eval. It runs your suite with the plugin loaded and again without it, then reports WITH, W/OUT, and Δ — the lift the plugin contributed. That changes what production teams should do before they merge a skill into a kit repo or pin it in CI.

This is not another “write better prompts” essay. It is a builder workflow for Claude Code v2.1.269+: create cases under evals/, grade outcomes, and fail the build when the score drops. If you already ship agent docs in an owned monorepo, pair this with Claude Code on an OTF kit: the CLAUDE.md and prompts that ship with the repo.

This is an excerpt. Read the full post at otf-kit.dev/blog/claude-code-plugin-eval-ci — full-stack kits your AI coding agent can actually ship to production. Browse the kits →

#agents #ai-tools #architecture

Comments

Join the discussion

No comments yet. Be the first to comment.

More from this blog

Paywall conversion for first paid users starts with one promise and one gate

First paid conversions climb when the paywall sells one concrete promise and the app enforces one entitlement gate — not when you stack every offer the stores allow on day one. Store subscription tool

Sep 17, 20261 min read

Paywall conversion for first paid users starts with one promise and one gate

Expo SDK 58 beta: test iOS 27 scenes and agent CLI before leaving 57

Expo published the SDK 58 beta on 15 September 2026. The beta runs three to four weeks — enough to learn whether iOS 27 scene lifecycle, React Native 0.88 RC, fingerprint defaults, and the new agent-f

Sep 17, 20261 min read

Expo SDK 58 beta: test iOS 27 scenes and agent CLI before leaving 57

Claude Opus 5.2 routing rumors: what builders should do before Anthropic confirms

Unannounced routing signals in Claude Code; use /status and metadata while shipping on Opus 5.

Sep 17, 20267 min read2

Claude Opus 5.2 routing rumors: what builders should do before Anthropic confirms

Mantine full suite vs assembling a thinner headless stack for web SaaS

Mantine is the rare component library that still feels like a product, not a grab bag. For web-only SaaS, the real question is no longer "is the component quality good enough?" — it is. The question i

Sep 17, 20261 min read

Mantine full suite vs assembling a thinner headless stack for web SaaS

Claude Opus 5.2 routing rumors: what builders should do before Anthropic confirms

If Claude Opus 5.2 lands as a mid-cycle refinement that cuts latency and fluff, that is a real builder tailwind. Faster answers and more persistence on long coding tasks matter more than a marketing r

Sep 17, 20261 min read

OTF — kits your AI coding agent can ship to production

599 posts

Engineering notes on shipping production apps with AI coding tools — Claude Code, Cursor, Codex, Lovable, Bolt — and the stack underneath: React Native, Expo, Next.js, Supabase. Honest takes on what works, what breaks, and the full-stack kits that get you to production faster. By OTF.

Command Palette

Comments

More from this blog