data-engineering
PluginData engineering plugin - warehouse exploration, pipeline authoring, Airflow integration
Install
claude plugin install data-engineering@claude-plugins-officialWhat is data-engineering?
Data engineering plugin - warehouse exploration, pipeline authoring, Airflow integration
What's inside
38 bundled components — installing the plugin installs all of them.
Skills (35)
airflow-adapter
Airflow adapter pattern for v2/v3 API compatibility. Use when working with adapters, version detection, or adding new API methods that need to work across Airflow 2.x and 3.x.
Name also used by astronomer-data-agents, data
airflow-hitl
Builds human-in-the-loop (HITL) Airflow workflows - approval gates, form input, and human-driven branching. Use when a DAG needs a human in the loop - an approval or reject step, sign-off before a task runs, a decision or approval UI, branching on a human choice, or collecting form input mid-run; al
Name also used by astronomer-data-agents, data
airflow-plugins
Builds Airflow 3.1+ plugins that embed FastAPI apps, custom UI pages, React components, middleware, macros, and operator links directly into the Airflow UI. Use when building anything custom inside Airflow 3.1+ that involves Python and a browser-facing interface - creating an Airflow plugin, adding
Name also used by astronomer-data-agents, data
airflow-state-store
Persists task and asset state across retries and DAG runs using Airflow 3.3's AIP-103 key/value stores (`task_state_store`, `asset_state_store`) and the crash-safe `ResumableJobMixin`. Use when the user asks about task state store, checkpointing in tasks, persisting state across retries, job IDs sur
Name also used by astronomer-data-agents, data
airflow
Queries, manages, and troubleshoots Apache Airflow using the `af` CLI. Use when working with anything related to Airflow - a DAG, a DAG run, a task log, an import or parse error, a broken DAG, or any Airflow operation. Covers listing and triggering DAGs, retrying runs, reading task logs, diagnosing
Name also used by astronomer-data-agents, data
analyzing-data
Queries the data warehouse with SQL and answers business questions about data. Use when answering anything that needs warehouse data - counts, metrics, trends, aggregations, joins across tables, data lookups, or ad-hoc SQL analysis (for example "who uses X", "how many Y", "show me Z", "find customer
Name also used by astronomer-data-agents, data
annotating-task-lineage
Annotate Airflow tasks with data lineage using inlets and outlets. Use when the user wants to add lineage metadata to tasks, specify input/output datasets, or enable lineage tracking for operators without built-in OpenLineage extraction.
Name also used by astronomer-data-agents, data
authoring-dags
Workflow and best practices for writing Apache Airflow DAGs. Use when creating a new DAG, write pipeline code, handling questions about DAG patterns and conventions or extending an existing DAG with a follow-up/downstream task. ANY request shaped like 'add a DAG named X', 'write a pipeline', 'add a
Name also used by astronomer-data-agents, data
authoring-go-sdk-tasks
Writes Airflow task logic in Go using the Airflow Go SDK. Use when the user wants to implement Airflow tasks in Go, asks about `BundleProvider`/`RegisterDags`, the `bundlev1` Registry/Dag interfaces, registering Go tasks (`AddTask`/`AddTaskWithName`), dependency injection by parameter type (`context
Name also used by astronomer-data-agents, data
authoring-java-sdk-tasks
Writes Airflow task logic in Java, Kotlin, or any JVM language using the Airflow Java SDK. Use when the user wants to implement Airflow tasks in Java/JVM, asks about `@Builder.Dag`/`@Builder.Task`/`@Builder.XCom`, the `Task`/`BundleBuilder` interfaces, reading connections/variables/XComs from Java,
Name also used by astronomer-data-agents, data
authoring-language-sdk-tasks
The language-neutral foundation for Airflow language SDKs — implement task logic in a non-Python language while the DAG stays in Python. Use when the user wants to run an Airflow task in another language (Java, Kotlin, Go, or other JVM/native languages), asks how the Python `@task.stub` pairs with n
Name also used by astronomer-data-agents, data
blueprint
Define reusable Airflow task group templates with Pydantic validation and compose DAGs from YAML. Use when creating blueprint templates, composing DAGs from YAML, validating configurations, or enabling no-code DAG authoring for non-engineers.
Name also used by astronomer-data-agents, data
checking-freshness
Quick data freshness check. Use when the user asks if data is up to date, when a table was last updated, if data is stale, or needs to verify data currency before using it.
Name also used by astronomer-data-agents, data
configuring-airflow-language-sdks
Configures Airflow to run language SDK tasks (Java, Go, and future native SDKs) — register a coordinator, map a queue to it, ensure the runtime/artifact on workers, and tune coordinator options. Use when the user wants Airflow to route a queue to a native-language coordinator, asks about the `[sdk]`
Name also used by astronomer-data-agents, data
cosmos-dbt-core
Turns a dbt Core project into an Airflow DAG/TaskGroup using Astronomer Cosmos. Use turning a dbt Core project into an Airflow DAG or TaskGroup with Astronomer Cosmos. Before implementing, verify dbt engine, warehouse, Airflow version, execution environment, DAG vs TaskGroup, and manifest availabili
Name also used by astronomer-data-agents, data
cosmos-dbt-fusion
Run a dbt Fusion project with Astronomer Cosmos. Use when running a dbt Fusion project with Astronomer Cosmos (Cosmos 1.11+, ExecutionMode.LOCAL on Snowflake/Databricks). Before implementing, verify dbt engine is Fusion (not Core), the warehouse is supported, and local execution is acceptable. Does
Name also used by astronomer-data-agents, data
creating-openlineage-extractors
Create custom OpenLineage extractors for Airflow operators. Use when the user needs lineage from unsupported or third-party operators, wants column-level lineage, or needs complex extraction logic beyond what inlets/outlets provide.
Name also used by astronomer-data-agents, data
dag-factory
Authors Apache Airflow DAGs declaratively from dag-factory YAML configs. Use when building DAGs declaratively from YAML via dag-factory; creating/editing dag-factory templates/YAML configs,reating/editing dag-factory YAML configs, defaults, dynamic tasks, datasets, or callbacks; or validating dag-fa
Name also used by astronomer-data-agents, data
debugging-dags
Comprehensive DAG failure diagnosis and root-cause analysis with structured investigation and prevention recommendations. Use when deep failure investigation is needed, a DAG fails to import/parse or 'airflow dags list' errors on a file; a task or run is failing and must be diagnosed and fixed; req
Name also used by astronomer-data-agents, data
delegating-to-otto
Drives Astronomer's Otto agent (`astro otto`) as a delegated sub-agent for Airflow, dbt, and data-engineering work. Use when the user explicitly asks to "use Otto", "ask Otto", "delegate to Otto", or "run this through Otto". Also offer Otto for Airflow 2 → 3 migrations and upgrade planning even when
Name also used by astronomer-data-agents, data
deploying-airflow
Deploys Airflow DAGs and projects. Use when deploying Airflow or answering anything about deployment - deploying DAGs/projects, pushing code, setting up CI/CD, deploying to production or deployment strategies for Airflow.
Name also used by astronomer-data-agents, data
deploying-go-sdk-bundles
Builds, packs, and deploys compiled Airflow Go SDK bundles so the ExecutableCoordinator can run them. Use when the user wants to compile a Go task bundle, asks about `go build`, `go tool airflow-go-pack`, the AFBNDL01 self-contained executable bundle, packing or inspecting a bundle, placing it under
Name also used by astronomer-data-agents, data
deploying-java-sdk-bundles
Builds and deploys compiled Airflow Java SDK bundles so workers can run them. Use when the user wants to package a JVM task bundle into a JAR, asks about the `org.apache.airflow.sdk` Gradle plugin, `./gradlew bundle`, the Maven shade/BOM setup, fat vs thin JARs, the logging integration artifacts (JP
Name also used by astronomer-data-agents, data
managing-astro-deployments
Manage Astronomer production deployments with Astro CLI. Use when the user wants to authenticate, switch workspaces, create/update/delete deployments, or deploy code to production.
Name also used by astronomer-data-agents, data
managing-astro-local-env
Manage local Airflow environment with Astro CLI (Docker and standalone modes). Use when the user wants to start, stop, or restart Airflow, view logs, query the Airflow API, troubleshoot, or fix environment issues. For project setup, see setting-up-astro-project.
Name also used by astronomer-data-agents, data
migrating-ai-sdk-to-common-ai
Migrates Airflow projects from airflow-ai-sdk to apache-airflow-providers-common-ai 0.4.0+. Use when replacing airflow-ai-sdk with the official Airflow AI provider - migrating LLM decorators (@task.llm, @task.agent, @task.llm_branch, @task.embed), switching from model strings/objects to connection-b
Name also used by astronomer-data-agents, data
migrating-airflow-2-to-3
Guide for migrating Apache Airflow 2.x projects to Airflow 3.x. Use when the user mentions Airflow 3 migration, upgrade, compatibility issues, breaking changes, or wants to modernize their Airflow codebase. If you detect Airflow 2.x code that needs migration, prompt the user and ask if they want you
Name also used by astronomer-data-agents, data
migrating-dagster-to-airflow
Guide for migrating Dagster projects to Apache Airflow 3 on Astro. Use when the user mentions migrating, converting, or porting Dagster (or Dagster+) code to Airflow or Astro, wants to plan or assess such a migration, or asks what a Dagster construct maps to in Airflow. Covers assets, partitions, sc
Name also used by astronomer-data-agents, data
profiling-tables
Deep-dive data profiling for a specific table. Use when the user asks to profile a table, wants statistics about a dataset, asks about data quality, or needs to understand a table's structure and content. Requires a table name.
Name also used by astronomer-data-agents, data
setting-up-astro-project
Initialize and configure Astro/Airflow projects. Use when the user wants to create a new project, set up dependencies, configure connections/variables, or understand project structure. For running the local environment, see managing-astro-local-env.
Name also used by astronomer-data-agents, data
testing-dags
Complex DAG testing workflows with debugging and fixing cycles. Use for multi-step testing requests like "test this dag and fix it if it fails", "test and debug", "run the pipeline and troubleshoot issues". For simple test requests ("test dag", "run dag"), the airflow entrypoint skill handles it dir
Name also used by astronomer-data-agents, data
tracing-downstream-lineage
Trace downstream data lineage and impact analysis. Use when the user asks what depends on this data, what breaks if something changes, downstream dependencies, or needs to assess change risk before modifying a table or DAG.
Name also used by astronomer-data-agents, data
tracing-upstream-lineage
Trace upstream data lineage. Use when the user asks where data comes from, what feeds a table, upstream dependencies, data sources, or needs to understand data origins.
Name also used by astronomer-data-agents, data
troubleshooting-astro-deployments
Troubleshoot Astronomer production deployments with Astro CLI. Use when investigating deployment issues, viewing production logs, analyzing failures, or managing deployment environment variables.
Name also used by astronomer-data-agents, data
warehouse-init
Initialize warehouse schema discovery. Generates .astro/warehouse.md with all table metadata for instant lookups. Run once per project, refresh when schema changes. Use when user says "/astronomer-data:warehouse-init" or asks to set up data discovery.
Name also used by astronomer-data-agents, data
Commands (3)
add-adapter-method
/add-adapter-methodAdd a new method to both Airflow adapters
Name also used by astronomer-data-agents, data
add-tool
/add-toolAdd a new MCP tool to server.py
Name also used by astronomer-data-agents, data
check-airflow-compat
/check-airflow-compatshellVerify code works with both Airflow 2.x and 3.x
Name also used by astronomer-data-agents, data
What this can do
Capabilities declared by the plugin's own components — read straight from their frontmatter, not inferred.
Run shell commands
1 component declare Bash
Inherit all session tools
37 components declare no tool restrictions
~3,598 tokens of context used while enabled, before you invoke anything
All declared tools (1)
BashTrust
87/100 · Excellent2 factors scored below maximum
Documentation
README · ~10 min readagents
AI agent tooling for data engineering workflows. Includes an MCP server for Airflow, a CLI tool (af) for interacting with Airflow from your terminal, and skills that extend AI coding agents with specialized capabilities for working with Airflow and data warehouses. Works with Claude Code, Cursor, and other agentic coding tools.
Built by Astronomer. Apache 2.0 licensed and compatible with open-source Apache Airflow.
Table of Contents
Reviews
Log in to leave a review.
No reviews yet — be the first.
Explore related
Other things in this space — across every part of the ecosystem, not just plugins.
Pluginssimilar to this one
All plugins →receipts
A personal Claude Code impact report for justifying your usage to a manager or a self-review: what you shipped, which projects it went to, and each project's share of your usage. Reads your ~/.claude/projects transcripts and runs read-only git locally; only counts and project names are sent to write
32.8K stars
atomic-agents
Comprehensive development workflow for building AI agents with the Atomic Agents framework. Includes specialized agents for schema design, architecture planning, code review, and tool development. Features guided workflows, progressive-disclosure skills, and best practice validation.
6.1K stars
databases-on-aws
Expert database guidance for the AWS database portfolio. Design schemas, execute queries, handle migrations, and choose the right database for your workload.
844 stars
MCP Servers
All mcp servers →Subagents
All subagents →explore
Read-only code explorer that the plugin's other agents dispatch to map a codebase — locate files, trace how a flow is wired, find every caller of a symbol, answer "where does X happen".
32.8K stars
scan-verifier
Restricted read-only verifier dispatched by the Claude Security scan workflow to vote on one candidate finding; not for direct invocation.
32.8K stars
scan-researcher
Restricted read-only vulnerability researcher dispatched by the Claude Security scan workflow; not for direct invocation or general exploration.
32.8K stars
Skills
All skills →create-atomic-tool
Build a `BaseTool[InSchema, OutSchema]` subclass — input/output schemas, `BaseToolConfig`, `run()` (and optional `run_async()`), env-driven secrets, typed failure outputs. Use when the user asks to "add a tool", "create a tool", "wrap an API as a tool", "build a `BaseTool`", "make a calculator/searc
6.1K stars
create-atomic-context-provider
Build a `BaseDynamicContextProvider` that injects a named, titled block into an agent's system prompt at every `run()` — current time, user identity, retrieved RAG docs, session state, cached DB schema. Use when the user asks to "add a context provider", "inject X into the prompt", "give the agent d
6.1K stars
framework
Guide for the Atomic Agents Python framework — schemas, agents, tools, context providers, prompts, orchestration, and provider configuration. Use when code imports from `atomic_agents`, defines an `AtomicAgent`, `BaseTool`, or `BaseIOSchema`, or the user asks about multi-agent orchestration or LLM-p
6.1K stars
Commands
All commands →altimate
Delegate a task to altimate-code, the specialised data-engineering CLI agent (warehouse access, column-level lineage, cross-DB, FinOps, query optimization)
windsor-types
Generate TypeScript type definitions for a Windsor.ai connector's data schema
design-survey
Interactive design session for a new survey — research goal, audience, hypotheses, questions, modality, output schema
Hooks
All hooks →PostToolUse (mcp__.*__find_columns|mcp__.*__get_dataset_columns|mcp__.*__get_dataset)
Pre-query column validation via schema caching. Catches unknown column errors before they hit the API by building a per-session schema cache from find_columns / get_dataset_columns / get_dataset results.
PreToolUse (mcp__.*__run_query)
Pre-query column validation via schema caching. Catches unknown column errors before they hit the API by building a per-session schema cache from find_columns / get_dataset_columns / get_dataset results.