Databricks, written down.
Every concept explained in full on its own page, not a link to somebody else's tutorial. It is one body of theory you can enter three ways — from the product area you are already in, along a path, or against an exam guide — and an agent watches the release notes so the pages say what the product says today.
What a lakehouse is, what Delta Lake does under it, and how Unity Catalog governs it — the three things every other page assumes you already know. After that the Data Engineer Associate roadmap is the most travelled route through the rest.
Learning paths
View allThe SQL and PySpark you need before any other path: the Spark dialect, types, joins, windows, MERGE, and the DataFrame API as it differs from pandas.
How the Databricks platform is put together: control plane and compute, Delta Lake as the storage format, Unity Catalog as the governance layer, and the medallion pattern everything else builds on.
Build production pipelines: ingest with Auto Loader, COPY INTO and Lakeflow Connect, transform with PySpark and SQL, orchestrate with Lakeflow Jobs, and ship it with bundles.
Unity Catalog end to end: the three-level namespace, managed and external tables, privileges, row filters, column masks and attribute-based policies.
Choose the right compute, read the Spark UI, tune the handful of settings that matter, and diagnose clusters that will not start or run out of memory.
Query the lakehouse from the SQL editor, model gold tables and views for BI, and understand what a SQL warehouse costs.
Latest news
All newsRecently updated
173 conceptsEvery concept is a markdown file
The whole site is written in a folder you can open in Obsidian: wikilinks resolve, the graph view works, callouts render. No plugins, nothing to configure.
- 1Download the zip and unzip it.
- 2In Obsidian choose "Open folder as vault".
- 3Pick the lakenaut folder. That is the whole setup.
Renamed recently
All renamesExam guides keep the old names for months. These are the ones that catch people out.
AI Search → Databricks AI Search2026-09Mosaic AI Gateway → Unity Gateway2026-08-04Genie spaces → Genie Agents2026-07Delta Sharing → OpenSharing2026-06-10In preview
The radarWhat Databricks is shipping but has not settled. Worth investigating, not worth betting on.
ABAC DENY policiesAttribute-based policies that deny rather than grant, evaluated against governed tags. BetaDashboard relationshipsA join graph across AI/BI dashboard datasets, resolved at query time, with cross-dataset measures over several fact tables. PreviewGoverned agentic app buildingDatabricks says you can now build governed data and AI apps without code. BetaEnding soon
All datesRuntimes, models and APIs with a date on them. Relevant if you have something in production.
The Supervisor APIThe Beta API for supervisor agents reaches end of life and stops being available.in 5 daysGemini 2.5 Flash and 2.5 ProBoth retire on the same day, Flash from pay-per-token and Pro from provisioned throughput.in 7 daysClaude Sonnet 4 on Foundation Model APIsRetired from pay-per-token serving. Requests to it will stop working.in 14 days