Resources

Self-paced courses are free with a Databricks account. The instructor-led ones are paid.

Walkthrough of a metastore, catalogs, schemas and grants in the UI.

The two incremental ingestion paths side by side, on the same data.

A declarative pipeline built and run end to end, expectations included.

How the task graph, triggers and dependencies fit together.

What Lakeflow actually does underneath the declarative syntax.

Why bundles exist and what they replace.

Past the getting-started example: variables, targets and real deployments.

Dataset to dashboard to Genie space, in one sitting.

The product thinking behind AI/BI, useful before you build one.

What serverless changes about sizing and cost.

How the warehouse decides to scale, and what that does to the bill.

An independent first look, with the caveats the launch post leaves out.

What Photon accelerates, what falls back, and whether it pays.

Clustering shown through the operation it helps most.

A conversation about where the layers go wrong, not a happy-path demo.

Index types, embeddings and when vector search is the wrong tool.

What changed in MLflow 3 and why runs and models were separated.

Authoring, evaluating and serving an agent, end to end.

Official channel: Data+AI Summit sessions, product deep dives, release walkthroughs.

Independent channel with hands-on Databricks walkthroughs and opinionated reviews of new features.

Metadata-driven bronze and silver pipelines: one specification per dataset, one generic Lakeflow pipeline that reads it. Formerly dlt-meta, and one of the few Labs projects with an official docs page.

Cost estimator for a workload before you run it: DBU sizing, cloud cost and configuration comparison, deployed as a Databricks App.

In-process Spark quality rules with quarantine and statistics tables, rules kept in a table and alerting to Kafka or email. The closest non-Databricks equivalent of DQX.

Python bindings for Deequ, "unit tests for data" on Spark: constraint suggestion, a metrics repository and anomaly detection over the metric history.

Checks written in YAML against warehouses and Spark, now positioned as a data contracts engine.

Expectation suites plus generated data documentation across Spark, pandas and SQL backends. Stewardship moved from the Great Expectations organisation to Fivetran in May 2026.

Schema and statistical typing for pandas, Polars and PySpark: declare a DataFrame schema and validate it in the job and in CI.

Time-series API on top of Spark: as-of joins, lagged values, resampling and rolling statistics without hand-written window functions.

Runs one operation across many Unity Catalog tables at once. Quiet since 2025 and part of its classification API is deprecated: useful as a script, not as a dependency.

Lightweight SQL execution over the Databricks SDK, for tools that need to run a query without a Spark session.

No longer actively maintained: its own README points to Databricks Asset Bundles. Worth knowing because plenty of repositories still deploy with it.

Toolkit for migrating a workspace onto Unity Catalog: assessment, table migration, rewriting jobs and dashboards.

Data quality checks on PySpark DataFrames and tables, batch and streaming, with quarantine and summary metrics. The programmatic counterpart to pipeline expectations.

Automates migration onto Databricks from other warehouses: source profiling, SQL dialect conversion and row-level reconciliation. Formerly called Remorph.

Generates synthetic data with Spark. The fastest way to get a realistic table to practise on.

Pytest fixtures for Databricks: real workspace objects created and cleaned up around a test.

The shared scaffolding behind the Labs projects: config, logging, CLI wiring for a Python project on Databricks.

The dbt adapter. Relevant if your transformations live in dbt rather than in pipelines.

Terraform for workspaces, clusters, policies and grants. Bundles ship code, Terraform provisions the platform.

Read and write Delta tables from Rust and Python without Spark. Useful for understanding the protocol.

The VS Code extension: sync a local folder, run a file on a cluster, debug with Databricks Connect.

The SQL connector for Python: query a warehouse from anything that is not a notebook.

Index of the Labs projects. Exploratory tools, not covered by a support agreement.

One-command demo notebooks for most Databricks features. The fastest way to see a concept running.

Source of the dbdemos installer and every demo notebook.

Reference implementations by industry. Useful for seeing a full pipeline rather than a snippet.

The CLI itself, including the bundle implementation. Read the issues when a bundle behaves oddly.

Working databricks.yml examples, from a one-job bundle to multi-target setups.

The Python SDK. The dataclasses in service/jobs.py are the clearest reference for job JSON.

Delta Lake itself. PROTOCOL.md is the authoritative description of the transaction log.

Upstream Spark. Worth opening when a Databricks doc says 'see the Spark documentation'.

Unofficial, MIT-licensed set of 71 Databricks product icons taken from official artwork, in SVG and PNG with Mermaid and draw.io packs. Handy for architecture diagrams; read the Databricks brand guidelines before public or commercial use.

Free ebook, registration required. Patterns rather than syntax.

Learning Spark, second edition. Free from Databricks with registration.

Delta Lake: The Definitive Guide. Free with registration.

Free online notes on Delta Lake internals. Deeper than the docs, occasionally behind them.

Free online notes on Spark SQL internals: Catalyst, physical plans, adaptive execution.

Official blog. Also one of the feeds the release-notes job reads.

Community forum. Often the only place an error message is explained.

Curated old-name to new-name catalogue. Also feeds the release-notes job.