ClaudeSuperPower
SubmitLog in

output-eval-dataset-design

Skill

Design diverse eval datasets using dimension-based variation. Use when bootstrapping eval datasets, when real traces are sparse, or when existing datasets miss edge cases.

0Machine Learning

Install

git clone https://github.com/growthxai/output.git ~/.claude/skills/output-eval-dataset-design

What is output-eval-dataset-design?

Design diverse eval datasets using dimension-based variation. Use when bootstrapping eval datasets, when real traces are sparse, or when existing datasets miss edge cases.

What this can do

Capabilities declared in this component's own frontmatter — not inferred.

Run shell commands

Declares Bash

Create and modify files

Declares Write or Edit

~43 tokens of context used while enabled, before you invoke anything

All declared tools (4)
BashEditReadWrite

Documentation

README · ~7 min read

Designing Eval Datasets

Overview

Diverse datasets catch more failures. This skill teaches dimension-based dataset design — systematically varying inputs along axes that target failure-prone regions of your workflow. The output is a set of YAML dataset files ready for output workflow test.

When to Use

  • Bootstrapping an eval dataset for a new workflow
  • Existing datasets only cover happy paths
  • Real production traces are sparse (fewer than 50)
  • Stress-testing specific failure hypotheses from error analysis

When NOT to Use

  • You already have 100+ representative real traces — use stratified sampling from those instead of generating synthetic data

Reviews

Log in to leave a review.

No reviews yet — be the first.

Explore related

Other things in this space — across every part of the ecosystem, not just skills.