ClaudeSuperPower
SubmitLog in

aidp-spark-debugging

Skill

Diagnose slow or failed AIDP Spark work using cluster logs, metrics, and the Spark UI REST API. Use when a job/query is slow or failed, the user asks "why did this fail / why is it slow", or you need stage/task timings, skew, shuffle/spill, executor, or SQL-execution details. Lightweight triage — fo

0Monitoring

Install

git clone https://github.com/oracle-samples/oracle-aidp-samples.git ~/.claude/skills/aidp-spark-debugging

What is aidp-spark-debugging?

Diagnose slow or failed AIDP Spark work using cluster logs, metrics, and the Spark UI REST API. Use when a job/query is slow or failed, the user asks "why did this fail / why is it slow", or you need stage/task timings, skew, shuffle/spill, executor, or SQL-execution details. Lightweight triage — for deep performance tuning (skew/spill/shuffle/joins/AQE/Delta) use the `aidp-spark-optimization` skill.

What this can do

Capabilities declared in this component's own frontmatter — not inferred.

Inherit all session tools

Declares no tool restrictions — inherits every session tool

~101 tokens of context used while enabled, before you invoke anything

Documentation

README · ~4 min read

aidp-spark-debugging — logs + metrics + Spark UI triage

Ground failure/slowness diagnosis in real execution data (never guess). Two engines, no MCP required:

  • Cluster logs & metrics → oci raw-request POST actions on the cluster (control-plane).
  • Spark UI job/stage/task/SQL detail → scripts/aidp_sql.py runs a kernel-side cell that hits the Spark REST API (spark.sparkContext.uiWebUrl + /api/v1/applications/...). The Spark UI is only reachable from inside the running kernel, so the helper is the no-MCP path for it.

When to use

  • A Spark job/query is slow or failed; "why did run X fail / why is it slow"; need stage/task/skew/shuffle detail.

Engines

1. Logs & metrics — oci raw-request (control-plane)

Reviews

Log in to leave a review.

No reviews yet — be the first.

Explore related

Other things in this space — across every part of the ecosystem, not just skills.

Subagents

All subagents →

code-architect

Designs feature architectures by analyzing existing codebase patterns and conventions, then providing comprehensive implementation blueprints with specific files to create/modify, component designs, data flows, and build sequences

32.8K stars

plugin-validator

Use this agent when the user asks to "validate my plugin", "check plugin structure", "verify plugin is correct", "validate plugin.json", "check plugin files", or mentions plugin validation. Also trigger proactively after user creates or modifies plugin components. Examples: <example> Context: User f

32.8K stars

code-explorer

Deeply analyzes existing codebase features by tracing execution paths, mapping architecture layers, understanding patterns and abstractions, and documenting dependencies to inform new development

32.8K stars