Jobs & Pipelines
PreviewWorth evaluating
Serverless job tasks can configure their environment from pyproject.toml (Beta)
From Databricks release notes (AWS)
Serverless notebook and Python script tasks in Lakeflow Jobs can now take their job environment from a pyproject.toml file, set in the Jobs UI, the Jobs API or a bundle. The feature is in Beta, and workspace admins enable it through Project-Scoped Shared Compute on the Previews page.
What to re-read
More from Jobs & Pipelines
- ReleaseLakeflow Jobs show job-level query performance metrics and insights (GA)
- ReleaseGenie Code tasks in Lakeflow Jobs are generally available
- ReleaseLakeflow system tables for pipelines reach general availability
- ReleaseSaving an unchanged job no longer writes rows to the jobs system tables