databricks-pipelines
SkillDevelop Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks. Use when building batch or streaming data pipelines with Python or SQL. Invoke BEFORE starting implementation.
Install
git clone https://github.com/databricks/databricks-agent-skills.git ~/.claude/skills/databricks-pipelinesWhat is databricks-pipelines?
Develop Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) on Databricks. Use when building batch or streaming data pipelines with Python or SQL. Invoke BEFORE starting implementation.
What this can do
Capabilities declared in this component's own frontmatter — not inferred.
Inherit all session tools
Declares no tool restrictions — inherits every session tool
~50 tokens of context used while enabled, before you invoke anything
Documentation
README · ~16 min readLakeflow Spark Declarative Pipelines Development
FIRST: Use the parent databricks-core skill for CLI basics, authentication, profile selection, and data discovery commands.
Decision Tree
Use this tree to determine which dataset type and features to use. Multiple features can apply to the same dataset — e.g., a Streaming Table can use Auto Loader for ingestion, Append Flows for fan-in, and Expectations for data quality. Choose the dataset type first, then layer on applicable features.
User request → What kind of output?
├── Intermediate/reusable logic (not persisted) → Temporary View
│ ├── Preprocessing/filtering before Auto CDC → Temporary View feeding CDC flow
│ ├── Shared intermediate streaming logic reused by multiple downstream tables
│ ├── Pipeline-private helper logic (not published to catalog)
Reviews
Log in to leave a review.
No reviews yet — be the first.
Explore related
Other things in this space — across every part of the ecosystem, not just skills.
Skillssimilar to this one
All skills →dbt-agent-skills
A curated collection of Agent Skills for working with dbt, to help AI agents understand and execute dbt workflows more effectively.
640 stars
authoring-language-sdk-tasks
The language-neutral foundation for Airflow language SDKs — implement task logic in a non-Python language while the DAG stays in Python. Use when the user wants to run an Airflow task in another language (Java, Kotlin, Go, or other JVM/native languages), asks how the Python `@task.stub` pairs with n
412 stars
airflow-adapter
Airflow adapter pattern for v2/v3 API compatibility. Use when working with adapters, version detection, or adding new API methods that need to work across Airflow 2.x and 3.x.
412 stars