ClaudeSuperPower
SubmitLog in

huggingface-community-evals

Skill

Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate. Not for HF Jobs orchestration, model-card PRs, .eval_results publication, or community-evals automation.

0Hardware

Install

git clone https://github.com/huggingface/skills.git ~/.claude/skills/huggingface-community-evals

What is huggingface-community-evals?

Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate. Not for HF Jobs orchestration, model-card PRs, .eval_results publication, or community-evals automation.

What this can do

Capabilities declared in this component's own frontmatter — not inferred.

Inherit all session tools

Declares no tool restrictions — inherits every session tool

~74 tokens of context used while enabled, before you invoke anything

Documentation

README · ~4 min read

Overview

This skill is for running evaluations against models on the Hugging Face Hub on local hardware.

It covers:

  • inspect-ai with local inference
  • lighteval with local inference
  • choosing between vllm, Hugging Face Transformers, and accelerate
  • smoke tests, task selection, and backend fallback strategy

It does not cover:

  • Hugging Face Jobs orchestration
  • model-card or model-index edits

Reviews

Log in to leave a review.

No reviews yet — be the first.

Explore related

Other things in this space — across every part of the ecosystem, not just skills.

Skillssimilar to this one

All skills →

m5-onboard

End-to-end onboarding for a freshly-plugged-in M5Stack ESP32 device (Cardputer, Cardputer-Adv, Core, CoreS3, Stick) — detect on USB, flash UIFlow 2.0 firmware, and install the Claude Buddy MicroPython app bundle. Use whenever the user plugs in or wants to flash/provision/reset an M5Stack or ESP32 bo

32.8K stars

cardputer-buddy

Iterate on the Cardputer-Adv MicroPython app bundle (Claude Buddy, Snake, Hello) after the device is already provisioned via m5-onboard. Use when the user wants to add a new app, push a single changed .py without re-flashing, watch device serial logs, or run a one-shot REPL command. Trigger on "add

32.8K stars

huggingface-spaces

Build, deploy, and maintain applications on Hugging Face Spaces — Gradio / Docker / Static SDKs, ZeroGPU and dedicated hardware, model loading, debugging, buckets, inference providers, community grants. Use whenever the user asks to create or host an app on Hugging Face, port code onto ZeroGPU, fix