Jobs & Pipelines
PreviewWorth evaluating
AI Function tasks arrive in Lakeflow Jobs (Beta)
From Databricks release notes (AWS)
Jobs can now include a task that runs ai_parse_document, ai_extract or ai_classify over files in a volume or rows in a table, with no SQL to write. Each run handles only new or changed inputs and writes results to a Unity Catalog table that downstream tasks can read. The page labels the feature Beta.
What to re-read
More from Jobs & Pipelines
- PreviewLakeflow Jobs can now start when upstream jobs finish (Beta)
- ReleaseLakeflow Jobs show job-level query performance metrics and insights (GA)
- PreviewServerless job tasks can configure their environment from pyproject.toml (Beta)
- ReleaseGenie Code tasks in Lakeflow Jobs are generally available