Jobs & Pipelines
ReleaseWorth evaluating
Maintenance windows can be set for continuous Lakeflow pipelines
From Databricks release notes (AWS)
You can now set a maintenance window on a continuous Lakeflow pipeline for platform updates that require a restart, including new runtime versions and serverless infrastructure maintenance. The window is a selected day, start hour, and time zone; Databricks starts maintenance only during that one-hour window and lets you configure it in the Jobs UI, the Pipelines UI, or the Jobs REST API.
Learners running continuous pipelines need to know that platform-update restarts can be scheduled instead of happening at an uncontrolled time.
What to re-read
More from Jobs & Pipelines
- ReleaseLakeflow Jobs show job-level query performance metrics and insights (GA)
- PreviewServerless job tasks can configure their environment from pyproject.toml (Beta)
- ReleaseGenie Code tasks in Lakeflow Jobs are generally available
- ReleaseLakeflow system tables for pipelines reach general availability