Data Ingestion
PreviewWorth evaluating
Data freshness column added to Lakeflow Connect database pipelines (Public Preview)
From Databricks release notes (AWS)
Lakeflow Connect database connector pipelines now show a Data freshness column: the time since the latest source commit ingested for each change data capture table. It works for SQL Server, PostgreSQL, MySQL and Oracle sources. Freshness and end-to-end lag can also be queried from the pipeline event log and used for alerts. In Public Preview.
What to re-read
More from Data Ingestion
- PreviewLakeflow Connect gets a managed Okta connector in Beta
- ReleaseZerobus Ingest into default-storage tables is generally available
- PreviewLakeflow Connect adds a Microsoft Graph Logs connector in Beta for security alerts
- ReleaseNative XML file format support reaches GA for Auto Loader, read_files and COPY INTO