output-eval-audit
SkillAudit an existing eval suite for trustworthiness. Use when inheriting evals, suspecting evals miss real failures, or after significant pipeline changes.
Install
git clone https://github.com/growthxai/output.git ~/.claude/skills/output-eval-auditWhat is output-eval-audit?
Audit an existing eval suite for trustworthiness. Use when inheriting evals, suspecting evals miss real failures, or after significant pipeline changes.
What this can do
Capabilities declared in this component's own frontmatter — not inferred.
Run shell commands
Declares Bash
~38 tokens of context used while enabled, before you invoke anything
All declared tools (2)
BashReadDocumentation
README · ~6 min readAuditing an Eval Suite
Overview
Audit your eval suite to determine whether it actually catches real failures. This skill provides a structured diagnostic that identifies gaps in error analysis, evaluator design, judge validation, and dataset coverage, with concrete remediation steps for each finding.
When to Use
- Inheriting an eval suite from another team or developer
- Suspecting that evals pass but production quality is poor
- After switching models, rewriting prompts, or changing pipeline logic
- Periodic health check (quarterly or after major releases)
Step 1: Gather Artifacts
Read the eval infrastructure files for the workflow being audited:
Reviews
Log in to leave a review.
No reviews yet — be the first.
Explore related
Other things in this space — across every part of the ecosystem, not just skills.
Skillssimilar to this one
All skills →ECC
The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.
235.3K stars
build-mcp-server
This skill should be used when the user asks to "build an MCP server", "create an MCP", "make an MCP integration", "wrap an API for Claude", "expose tools to Claude", "make an MCP app", or discusses building something with the Model Context Protocol. It is the entry point for MCP server development
32.8K stars
claude-md-improver
Audit and improve CLAUDE.md files in repositories. Use when user asks to check, audit, update, improve, or fix CLAUDE.md files. Scans for all CLAUDE.md files, evaluates quality against templates, outputs quality report, then makes targeted updates. Also use when the user mentions "CLAUDE.md maintena
32.8K stars
MCP Servers
All mcp servers →Commands
All commands →modernize-harden
Security vulnerability scan with a reviewable remediation patch — OWASP, CWE, CVE, secrets, injection
32.8K stars
modernize-status
Where am I in the modernization workflow — artifact inventory, staleness, secrets hygiene, next step
32.8K stars
create-docker-mcp-tunnel
Stand up an Anthropic MCP tunnel locally with Docker Compose so Claude can call a private MCP server (manual-credentials quickstart).
32.8K stars
Plugins
All plugins →claude-security
Deep vulnerability scanning of your own code, run entirely inside your Claude Code session at a chosen effort tier, with every finding challenged before it is reported and the verification tally computed in code. Turns surviving findings into targeted patches, each verified by a panel of agents, tha
32.9K stars
claude-md-management
Tools to maintain and improve CLAUDE.md files - audit quality, capture session learnings, and keep project memory current.
32.9K stars
mcp-tunnels
Connect Claude to a private MCP server through an Anthropic MCP tunnel. The /create-docker-mcp-tunnel command drives the Docker Compose quickstart end to end: certificates, proxy config, cloudflared, and a verifiable sample server.
32.9K stars
Subagents
All subagents →ECC
The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.
235.3K stars
claude-security
The dedicated Claude Security orchestrator. Hand it an unattended job — "fully scan this repository and patch what you find; I understand it will use a lot of tokens" — and it runs the whole thing itself: capturing the revision, driving the multi-agent scan through the claude-security:scan workflow,
32.8K stars
scan-verifier
Restricted read-only verifier dispatched by the Claude Security scan workflow to vote on one candidate finding; not for direct invocation.
32.8K stars