Streaming
ReleaseWorth evaluating
Asynchronous processing for Python transformWithState is generally available
From Databricks release notes (AWS)
The Python row-based transformWithState API can now run state operations and user logic concurrently, which the release note says raises throughput with only minor code changes. The feature is generally available.
What to re-read
More from Streaming
- ReleaseOn-demand state repartitioning is generally available for stateful streaming
- ReleaseOn-demand state repartitioning added for Structured Streaming
- PreviewManaged tables can be created in Lakeflow pipelines in Beta