Databricks news

ReleaseData IngestionVariant shredding will be added by default to new VARIANT tablesStarting November 18, 2026, Databricks will automatically add the variantShredding table feature to new tables with a VARIANT column when they are created on Databricks Runtime 17.3 or above. Variant shredding stores frequently accessed fields in a typed, columnar layout to improve query performance. Clusters on Databricks Runtime 15.4 cannot read from or write to those tables, so affected clusters should be upgraded to Databricks Runtime 16.4 or above before the change takes effect.

ReleaseJobs & PipelinesJAR tasks on serverless compute are coming to compliance-profile workspacesIn early October 2026, Databricks will automatically enable JAR tasks on serverless compute for workspaces with the compliance security profile enabled. Users can package Java or Scala Spark code in a JAR and run it as a JAR task in a Lakeflow Job without provisioning or managing clusters.

ReleaseComputeEnvironment version 6 is the new serverless baseThe base environment for serverless and standard classic compute moved to version 6. Pinning an older one keeps working, but the default a new workload gets has changed.

News