Reading the query profile

kept in this browser

The query profile turns a finished statement into a graph of operators and metrics for spotting scans, spills, and bad joins.

Checked against the Databricks documentation on · history

On this page10

What it is

Query History lists every statement run on a SQL warehouse, with duration, user, and status. Opening one and clicking See query profile shows a query profile: a directed graph of the operators the engine actually executed — scans, joins, aggregations, shuffles — each annotated with its own metrics. It is the SQL-warehouse equivalent of the Spark UI’s stages and tasks (see Spark UI: skew, shuffle, and spill), but built around the operators a SQL statement compiles into rather than raw Spark stages.

Why it exists

A query duration tells you that something is slow, not what. The profile graph attributes time, rows, and bytes to each individual operator, so instead of guessing you can point at the one join or scan responsible for most of the runtime and fix that specific thing.

How it works

The graph

The profile renders the query plan as a DAG: each node is an operator, edges show data flowing between them. Clicking a node opens its detailed metrics; a side panel also summarizes the three tabs Details (overall stats), Top operators (the most expensive ones), and Query text.

The metrics that matter

MetricWhat it tells you
Rows read vs. rows returneda huge gap on a scan usually means a missing filter or missing pruning
Bytes readhow much data actually had to be touched — compare it to the table’s total size
Files/partitions prunedhow much of the table the engine skipped based on filters and file statistics
Spill to diskthe operator’s working set didn’t fit in memory — undersized warehouse, skew, or an exploding join
Shufflerows moved between workers to co-locate data for a join or aggregation — expensive, and worse when one side is much larger than the other
Time spent / memory peakwhich operator to optimize first

Spotting a missing filter

If a scan’s rows read is close to the table’s full row count while rows returned is tiny, the predicate isn’t being pushed down or pruning isn’t happening — check that the filter is on a clustering or partition column, and that it isn’t wrapped in a function that defeats pruning (WHERE YEAR(order_date) = 2026 instead of a direct range on order_date).

Spotting a bad join

A join with a much larger rows out than the sum of its inputs is an exploding join: a many-to-many match on a key that should be unique on at least one side. Heavy shuffle and spill on a join usually mean the smaller side didn’t get broadcast, or one join key is heavily skewed.

Photon

Photon-executed operators report their own metrics; some non-Photon operators are grouped together and share combined metrics rather than being broken out individually, so a profile from a Photon-enabled warehouse can look coarser in places than a fully vectorized breakdown.

Example

Two versions of the same aggregation — compare their profiles rather than their SQL:

-- 1) No filter: full scan, rows read ≈ table size, no pruning
SELECT customer_id, SUM(amount) AS total
FROM sales.gold.orders
GROUP BY customer_id;
-- 2) Filtered on the clustering column: files pruned, rows read shrinks
SELECT customer_id, SUM(amount) AS total
FROM sales.gold.orders
WHERE order_date >= DATE'2026-01-01'
GROUP BY customer_id;

In the first profile, the scan operator shows most files read and no pruning; in the second, the same scan shows a high pruning percentage and a much smaller bytes read, with less downstream work for the aggregation.

Common mistakes

  • Looking only at total query duration instead of which operator owns most of the time.
  • Ignoring the gap between rows read and rows returned — the clearest sign of a missing or ineffective filter.
  • Not checking spill to disk, then concluding the query is “just slow” instead of undersized or skewed.
  • Treating queueing time (waiting for a free cluster, see Sizing a SQL warehouse) as part of the query’s own execution time.
  • Assuming Photon’s grouped metrics mean an operator did nothing, when it’s simply reported together with neighboring steps.

Check yourself

3 questions on what this page says. Pick an answer, then check it.

  1. 1In a query profile, a scan shows rows read close to the table's full row count while rows returned is tiny. What does the page say this points to?
    Show the answer

    C. A missing or ineffective filter, where the predicate is not pushed down or pruning does not happen

    A big gap between rows read and rows returned on a scan is the clearest sign that the filter is missing or not being used for pruning, for example because it is wrapped in a function.

    Read the section again
  2. 2A join in the profile has a much larger rows out than the sum of its two inputs. What does the page call this, and what is the usual cause?
    Show the answer

    B. An exploding join, caused by a many-to-many match on a key that should be unique on at least one side

    Output far larger than the inputs is an exploding join, which comes from matching on a key that is not unique on either side. Resizing the warehouse does not remove the problem.

    Read the section again
  3. 3A query is slow, and someone proposes a larger warehouse. What does the page advise doing first?
    Show the answer

    A. Open the query profile, because a scan with no pruning or an exploding join stays slow on a bigger warehouse and only costs more DBUs

    The profile attributes time and data to each operator, so the real cause is visible before spending more. The page warns that a bigger warehouse does not fix pruning or join problems.

    Read the section again

Where this sits

Resources

2All resources
Report a problem with this page
What kind of problem?

Reports about "Reading the query profile" go to the maintainer, not to a public thread.

Suggest a resource
What kind?

Nothing appears on the site automatically. A person reads every suggestion, checks the link and writes the note that goes with it.