Jobs & Pipelines
PreviewWorth evaluating
Lakeflow Jobs can now start when upstream jobs finish (Beta)
From Databricks release notes (AWS)
A Lakeflow Job can be set to run when one or more upstream jobs complete, including jobs in other workspaces attached to the same Unity Catalog metastore. The dependency is configured on the downstream job. The feature is in Beta.
What to re-read
More from Jobs & Pipelines
- ReleaseLakeflow Jobs show job-level query performance metrics and insights (GA)
- PreviewServerless job tasks can configure their environment from pyproject.toml (Beta)
- ReleaseGenie Code tasks in Lakeflow Jobs are generally available
- ReleaseLakeflow system tables for pipelines reach general availability