ClaudeSuperPower
SubmitLog in

scrape

Skill

End-to-end web scraping workflow — from URL to working spider with web-poet page objects. Use this for full-site or multiple-page crawls that create a new scraping workflow. Do not use for fixing, debugging, or modifying an existing spider.

0Automation

Install

git clone https://github.com/zytedata/claude-skills.git ~/.claude/skills/scrape

What is scrape?

End-to-end web scraping workflow — from URL to working spider with web-poet page objects. Use this for full-site or multiple-page crawls that create a new scraping workflow. Do not use for fixing, debugging, or modifying an existing spider.

What this can do

Capabilities declared in this component's own frontmatter — not inferred.

Run shell commands

Declares Bash

Create and modify files

Declares Write or Edit

~60 tokens of context used while enabled, before you invoke anything

All declared tools (9)
AgentBashReadSkillTaskCreateTaskGetTaskListTaskUpdateWrite

Documentation

README · ~3 min read

You are orchestrating the full web scraping workflow, from a user's prompt to a working Scrapy spider with web-poet page objects.

Prerequisites

Requires uv. Install if missing.

Input

The raw argument string is $ARGUMENTS. Split it into 2 positional arguments:

  1. url: target website URL (first whitespace-separated token)
  2. what: what the user wants to extract (the rest after the URL, e.g. "product", "job listing", "recipe" — free text, may contain spaces)

Track progress

Before Stage 1, create exactly these tasks with TaskCreate, in order:

Reviews

Log in to leave a review.

No reviews yet — be the first.

Explore related

Other things in this space — across every part of the ecosystem, not just skills.

Skillssimilar to this one

All skills →

link-rot-scanner

Crawl and validate all internal and external links across an AEM Edge Delivery Services site. Uses the query index or sitemap to discover pages, extracts links from .plain.html renditions, checks HTTP status codes, and produces a prioritized report of broken, redirecting, and insecure links. Use whe

151 stars

extract

Crawl an existing website (capped, multi-page) and seed stardust/current/ with PRODUCT.md, DESIGN.md, DESIGN.json, a per-page inventory, and the consolidated brand surface — the captured design system, palette, typography, motifs, and voice of the live site. Use when the user wants to analyze an exi

151 stars

scrape-webpage

Use this when the page-import pipeline needs to fetch a source webpage and prepare it for import/migration to AEM Edge Delivery Services. Covers scraping content, extracting metadata, downloading images, and returning analysis JSON with paths, metadata, cleaned HTML, and local images. Do not invoke

151 stars