ClaudeSuperPower
SubmitLog in

scrape-create-spider

Skill

Generate a Scrapy spider that wires page objects together

0Other

Install

git clone https://github.com/zytedata/claude-skills.git ~/.claude/skills/scrape-create-spider

What is scrape-create-spider?

Generate a Scrapy spider that wires page objects together

What this can do

Capabilities declared in this component's own frontmatter — not inferred.

Run shell commands

Declares Bash

Create and modify files

Declares Write or Edit

~14 tokens of context used while enabled, before you invoke anything

All declared tools (4)
BashEditReadWrite

Documentation

README · ~3 min read

You are generating a Scrapy spider that wires together web-poet page objects (item extraction + navigation) into a working crawler.

Read python-environments.md and docs-access.md from ${CLAUDE_SKILL_DIR}/../scrape/references.

Input

The raw argument string is $ARGUMENTS. Split it into 3 whitespace-separated positional arguments:

  1. project_dir: path to the Scrapy project
  2. item_page: import path of the item extraction PO (e.g. books_project.pages.books_toscrape_com.ProductPage)
  3. nav_page: import path of the navigation PO (e.g. books_project.pages.books_toscrape_com.NavigationPage)

Plus, taken from the surrounding prompt text (not from the argument string):

Reviews

Log in to leave a review.

No reviews yet — be the first.

Explore related

Other things in this space — across every part of the ecosystem, not just skills.

Skillssimilar to this one

All skills →

link-rot-scanner

Crawl and validate all internal and external links across an AEM Edge Delivery Services site. Uses the query index or sitemap to discover pages, extracts links from .plain.html renditions, checks HTTP status codes, and produces a prioritized report of broken, redirecting, and insecure links. Use whe

151 stars

extract

Crawl an existing website (capped, multi-page) and seed stardust/current/ with PRODUCT.md, DESIGN.md, DESIGN.json, a per-page inventory, and the consolidated brand surface — the captured design system, palette, typography, motifs, and voice of the live site. Use when the user wants to analyze an exi

151 stars

scrape-webpage

Use this when the page-import pipeline needs to fetch a source webpage and prepare it for import/migration to AEM Edge Delivery Services. Covers scraping content, extracting metadata, downloading images, and returning analysis JSON with paths, metadata, cleaned HTML, and local images. Do not invoke

151 stars