Databricks news

Yesterday27 Oct 2026

ReleaseCatalogUnity Catalog permissions on system.ai now control which AI Functions users can run (GA)Metastore admins can use Unity Catalog permissions on the system.ai schema to decide which task-specific AI Functions people in the organization may run. The feature is generally available, but the page says to contact your Databricks account team to enable it.
ReleaseAgentsDefault OAuth client IDs let ChatGPT, Claude, Cursor and others reach Databricks MCP serversChatGPT, Codex, Claude, Claude Code and Cursor can now connect to Databricks MCP servers, including the Genie One MCP server, with default OAuth client IDs. An account admin no longer has to create an OAuth app first. The feature is generally available.
ReleaseWorkspaceUnity Catalog volume files can be edited in the workspace file editor (GA)Files stored in Unity Catalog volumes can now be opened and edited in place in the workspace file editor, with no download and re-upload. You open the file from Catalog Explorer or from the schema browser in a notebook or the SQL editor. The feature is generally available.

Monday 26 October26 Oct 2026

Saturday 24 October24 Oct 2026

PreviewData IngestionLakeflow Connect adds a managed PayPal connector in BetaA managed PayPal connector is in Beta in Lakeflow Connect. It ingests payments data such as transactions, disputes, invoices, balances, catalog products and billing plans. It uses OAuth client credentials and supports incremental ingestion.

Friday 23 October23 Oct 2026

Thursday 22 October22 Oct 2026

Wednesday 21 October21 Oct 2026

Tuesday 20 October20 Oct 2026

PreviewUnity Gatewayai_enrich can ground each row with an external MCP search server (Beta)The ai_enrich function can now ground every row with a search tool on an external Model Context Protocol server registered in Unity Gateway, such as You.com or Exa. An mcp knowledge source can be used alone or next to other knowledge sources in one call. It is in Beta, and workspace admins enable it from the Previews page.

Monday 19 October19 Oct 2026

Friday 16 October16 Oct 2026

Thursday 15 October15 Oct 2026

Wednesday 14 October14 Oct 2026

ReleaseCatalogData Classification API and Terraform resource are generally availableThe Data Classification API and its Terraform resource have reached general availability. The auto-tagging configuration (auto_tag_configs) is still in Public Preview.
ReleaseSQLDocument Intelligence REST APIs for parse, extract and classify reach GASynchronous REST APIs for ai_parse_document, ai_extract and ai_classify are generally available, so an app or agent can process a file as it arrives. The ai_decide API, used for routing documents, is in Beta. Databricks Apps and agents built on Agent Bricks are the intended callers.
PreviewSQLPredictive AI Functions enter Beta: forecasting, anomalies and predictions in SQLPredictive AI Functions are in Beta: an upgraded ai_forecast plus new ai_top_drivers, ai_detect_anomalies, ai_predict_value and ai_predict_class, all callable from SQL. ai_forecast gains holiday support and external covariates. Genie can already call ai_top_drivers; the other four are listed as coming to Genie later.
PreviewComputePrivate Network Gateway for serverless: Private Preview on AWS, Public Preview on AzurePrivate Network Gateway lets serverless workloads reach resources in your own network through one delegated subnet, configured in a Network Connectivity Configuration. On AWS it is in Private Preview, with enrollment through the account team; on Azure it is in Public Preview. Google Cloud support is described as in development.

Tuesday 13 October13 Oct 2026

ReleaseDiscoverClean rooms: one-step packaged clean room with an identity resolution provider is GAFrom the Clean Rooms page you can now create a ready-made clean room with an approved identity resolution provider, such as Acxiom, Adstra, LiveRamp or The Trade Desk, in a single step. The provider is already set as the package provider, so you add your data and run its analysis straight away.
ReleaseDiscoverClean rooms: shared output schema is GAClean room runs can write to a shared output schema that every collaborator can read as each table is created, even before the run finishes. It is meant for sharing results, logs and debugging information.
ReleaseDiscoverClean rooms: JAR workloads are GAClean rooms can now run compiled Java or Scala code. A JAR runs as its own clean room analysis, or a function registered from a JAR can be called as a temporary UDF from a clean room notebook.
ReleaseDelta LakeExternal Spark clients can write to Unity Catalog managed Delta tables (GA)Creating and writing Unity Catalog managed Delta tables from external Apache Spark clients is now generally available. The tables must have catalog commits enabled.
ReleaseCatalogExternal engines can query Unity Catalog views via the Unity REST API (GA)External engines can now read Unity Catalog views defined over Delta Lake tables through the Unity REST API, and the capability is generally available. Teams that expose governed data to non-Databricks engines can include views in that access path instead of only tables.
ReleaseStreamingOn-demand state repartitioning is generally available for stateful streamingYou can now change the number of shuffle partitions of a stateful Structured Streaming query or a Lakeflow pipeline without losing its checkpoint state. The feature is generally available.
ReleaseUnity GatewayGemini 4 Argon is now available on Unity Gateway through Foundation Model APIsGoogle's Gemini 4 Argon model can now be reached through Unity Gateway using Foundation Model APIs. The release note says customers remain responsible for complying with the applicable model terms.

Monday 12 October12 Oct 2026

ReleaseComputePrivate Link from serverless compute to AWS-managed resources is generally availablePrivate connectivity from Databricks serverless compute to AWS-managed resources such as S3 buckets, previously in Public Preview, is now generally available. Traffic stays inside the AWS network instead of crossing the public internet.
PreviewAgentsAgent memory pipelines distil conversations into agent memory (Beta)A memory pipeline links a session store to a memory store. Each run uses an LLM to extract durable facts, preferences and decisions from stored conversations into memory entries that an agent can recall later. Pipelines run on demand or about every 24 hours, and the feature is in Beta.
PreviewCatalogGoverned tags can now propagate through lineage to derived tables (Beta)With tag propagation enabled on a governed tag, Unity Catalog uses lineage to apply that tag to derived tables and views after supported operations such as CREATE TABLE AS SELECT, CREATE VIEW, INSERT INTO ... SELECT and MERGE. The aim is consistent classification and ABAC enforcement downstream, at table and column level. It is in Beta.
PreviewUnity GatewayUnity Gateway Skills REST API enters Beta with CLI and SDK supportA REST API for managing skills on Unity Gateway is now in Beta, covering create, read, update, list, delete and finalize. The Databricks CLI and the Python, Go, Go Modular, Java and JavaScript SDKs support it.
ReleaseJobs & PipelinesLakeflow pipeline parameters are generally available for SQL sourcesKey-value parameters for Lakeflow pipelines have left preview. One pipeline's SQL source can run against different catalogs, schemas or tables: defaults live in the pipeline settings, an update can override them, and the SQL refers to them with named parameter syntax.
ReleaseFeaturesFeature Views reach general availability for batch, streaming and request-time dataFeature Views, the declarative way to define and compute features, are generally available for batch, streaming and request-time data. For workspaces with the compliance security profile, they will be switched on automatically in mid-November 2026.

Friday 9 October9 Oct 2026

ReleaseAppsDatabricks Apps: workspace admins can set default Python and npm package repositoriesWorkspace-level package repositories for Databricks Apps are generally available. Admins set default Python and npm repositories that every app inherits, and an app can still override them. A separate upcoming note says that from November 13, 2026 the feature is turned on automatically for workspaces with the compliance security profile.
ReleaseJDBC Unity Catalog connections to be on by default under the compliance profileIn mid-November 2026, JDBC Unity Catalog connections, including SSL certificate support, will be available by default for workspaces with the compliance security profile enabled. Such a connection reaches an external database through a custom JDBC driver, with access governed by Unity Catalog. The note gives a month, not a day.
ReleaseUnity GatewayUnity Gateway model provider services can reach Amazon Bedrock Mantle modelsAmazon Bedrock Mantle models can now be called through Unity Gateway model provider services. The calls are governed by Unity Catalog permissions, rate limits and budgets, and are recorded in inference tables.
PreviewGenieGenie One can run code in an isolated sandbox (Public Preview)Genie One can now run code in a secure, isolated sandbox for analyses that go beyond SQL. The code runs with the user's existing permissions, so Unity Catalog governance and data access controls still apply. The feature is in Public Preview.
PreviewUnity GatewayOpenJev can be the evaluator model for LLM-as-a-judge service policies (Beta)In Beta, a custom LLM-as-a-judge service policy on Unity Gateway can use OpenJev (Qwen3.5 4B) as its evaluator model service. The page says OpenJev classifies content without generating output tokens, so it typically responds faster than a chat evaluator.
PreviewUnity GatewayUnity Gateway service policies can require caller-supplied request tags (Beta)In Beta, a custom service policy on Unity Gateway can check tags that the caller attaches to a request, for example requiring a given tag before a model request or an MCP tool call goes through. This lets a policy decide on caller context rather than only on content.

Thursday 8 October8 Oct 2026

PreviewData IngestionGoogle Workspace connector for Lakeflow Connect reaches BetaLakeflow Connect has a managed Google Workspace connector in Beta. It ingests audit activity from Google Workspace applications and services into Databricks, using OAuth user authorization and incremental ingestion.
PreviewData IngestionProductboard connector for Lakeflow Connect is in BetaLakeflow Connect has a managed Productboard connector in Beta. It ingests product hierarchy, roadmap, objectives, feedback notes and workspace members, using Server-to-Server OAuth and incremental ingestion for notes. Workspace admins control access from the Previews page.
ReleaseUnity Catalog Scala, Java and Python UDFs to be on by default under the compliance profileUnity Catalog Scala and Java UDFs, and scalar and batch Unity Catalog Python UDFs, will be available by default in November 2026 for workspaces with the compliance security profile enabled. The note gives the month only, not a day.
PreviewGenieGenie One file upload accepts plaintext files such as .txt and Markdown (Beta)File upload in Genie One now also takes plaintext files, including .txt and Markdown (.md), alongside CSV, Excel, PDF, image, Word and PowerPoint files. The release note labels this addition Beta; it does not announce general availability.
ReleaseCatalogThe Discover page and domains in Unity Catalog reach general availabilityDiscover, a curated way to browse assets without knowing the catalog hierarchy, and domains with subdomains are now generally available. Domains group tables, dashboards, notebooks and Genie Agents by business area, allow curated highlights, and surface AI-recommended assets.
PreviewCatalogWorkday Data Connect federation connector for Unity Catalog opens in BetaA Lakehouse Federation connector lets Unity Catalog read Workday's shared Iceberg tables in place, with no ingestion pipeline and read-only access. It needs Databricks Runtime 19 or above, and a workspace admin must enable it from the Previews page. Lakeflow Connect for Workday remains the route when a durable copy is needed.

Wednesday 7 October7 Oct 2026

DeprecatedDeepSeek V4 Flash (0731) will be retired on 5 November 2026
ReleaseContext-based ingress policies for inbound Private Link are generally availableContext-based ingress policies for inbound Private Link have reached general availability. They control which registered VPC endpoints can reach workspaces and combine endpoint rules with identity and request-type conditions. If the feature is not yet enabled for an account, an account admin can turn it on from the account console Previews page.
PreviewDashboardsAI/BI dashboards can pre-compute dataset results on a schedule (Beta)Dataset materialization for AI/BI dashboards is in Beta. Dataset results are computed on a schedule and stored, so a published dashboard renders from the stored results instead of rerunning expensive queries on every load or interaction.
PreviewUnity GatewayUnity Gateway external service policies send traffic to a partner guardrail (Beta)In Beta, an external service policy on Unity Gateway forwards request and response content to a third-party guardrail vendor and blocks the call if the vendor denies it. Applications need no changes.
ReleaseAppsOn-behalf-of-user authorization for Databricks Apps is generally availableDatabricks Apps can now call supported APIs as the signed-in user, so Unity Catalog row filters and column masks apply to that user, while API scopes cap what the app may do. A read-only query app can request sql:restricted-query rather than the broader sql scope, and workspace admins can restrict which scopes developers may add.
ReleaseServingDeepSeek V4.1 Flash gains priority pay-per-token and reserved throughputDeepSeek V4.1 Flash can now be served with priority pay-per-token for latency-sensitive traffic, or with reserved provisioned throughput for dedicated capacity.
ReleaseUnity GatewayClaude Haiku 5.5 is available on Unity Gateway through Foundation Model APIsAnthropic's Claude Haiku 5.5 can now be used on Unity Gateway via Foundation Model APIs. The note reminds customers that they are responsible for complying with the applicable terms.

Tuesday 6 October6 Oct 2026

ReleaseAction requiredCatalogCREATE OR REPLACE VIEW will keep grants, tags and comments from 2 December 2026From December 2, 2026, CREATE OR REPLACE VIEW will update a view in place instead of dropping and recreating it, so grants, tags and comments are preserved. Governed tags on a view column must be removed before the column is dropped this way, otherwise the statement fails with CANNOT_DROP_TAGGED_COLUMN. Adding NOCOPY drops grants, tags and comments explicitly, though governed tags are still retained.
PreviewGenieGenie Code CLI, a terminal coding agent for Databricks, is in BetaGenie Code CLI is a coding agent that runs in a terminal and works on local files. It uses the Databricks CLI to find data, answer questions, and build and deploy pipelines, models and apps. Model access goes through Unity Gateway. It is labelled Beta.
ReleaseCatalogWindow measures in metric views reach general availabilityMetric views can now define window measures, which cover rolling windows, cumulative totals and semiadditive aggregations such as moving averages, period-over-period change and running totals. The feature is generally available as of this note. Anyone modelling time-based KPIs in metric views no longer has to treat it as a preview.

Monday 5 October5 Oct 2026

DeprecatedDatabricks Runtime 10.4 LTS ends on 2026-11-01
DeprecatedGenie One MCP Beta endpoint (/api/2.0/mcp/genie) ends on 2026-10-31
DeprecatedDeepSeek V4 Pro (0813) ends on 2026-10-30
ReleaseSQLai_parse_document now returns chart values as structured JSONThe ai_parse_document function extracts chart values as structured JSON and describes each chart's series, legends and axes. Search and agents can then use those values when answering.
PreviewGenieGenie One skills can be shared and governed through Unity Gateway (Beta)A skill built in Genie One can now be published and shared with a team in Beta. Access is governed by Unity Gateway and uses ordinary Unity Catalog privileges on the catalog and schema that hold the skill.
PreviewJobs & PipelinesLakeflow Jobs can use up to five triggers per job (Beta)A Lakeflow job can now carry up to five triggers. Trigger types can be mixed, several triggers of the same type are allowed, and each trigger can be paused on its own. The release note labels this Beta.
ReleaseAI SearchNEAREST BY: a SQL join for batch vector search in Databricks RuntimeNEAREST BY is a SQL join that finds, for each row on the left, the k closest rows on the right by vector similarity or distance, either EXACT or APPROX. It runs in Photon with a fused operator, and APPROX queries can use an IVF index stored as a liquid-clustered Delta table. The post does not state a release stage, so availability needs checking against the documentation.
PreviewCatalogIceberg REST catalogs gain UNREGISTER; Unity Catalog support is in private previewDatabricks contributed an UNREGISTER endpoint to the Apache Iceberg REST catalog specification. It removes a table from one catalog without touching its files and returns the latest metadata location, which another catalog then takes over with REGISTER, so a table has one managing catalog throughout the move. REGISTER and UNREGISTER on Unity Catalog are in private preview and available through the account team.

Friday 2 October2 Oct 2026

ReleaseJobs & PipelinesLakeflow pipelines can run as an account groupA Lakeflow pipeline's run-as identity can now be an account group, in addition to a user or a service principal. Pipeline updates run with the group's permissions, so a team can own an automated pipeline without it depending on one person's account.
ReleaseQuery HistoryCustomer-managed keys can now decrypt query history text fields (GA)A customer-managed key set on the system catalog can now be used to decrypt the statement_text and error_message fields in the query history system table. The release notes mark this generally available as of 2 October 2026. Teams that encrypt query history with their own key should read the docs section on reading encrypted fields.
DeprecatedGemini 2.5 Pro and 2.5 Flash retired; Gemini 3.1 Pro or 3.5 Flash replace them
ReleaseUnity GatewayOpenJev (Qwen3.5 4B) is available on Unity GatewayOpenJev (Qwen3.5 4B) is now offered on Unity Gateway as a pay-per-token model service. The release notes say it is meant for yes/no, choice and scoring questions that return structured answers with probabilities, and that customers must check compliance with the applicable model terms.
ReleaseSQLStandard ai_extract moves to a faster model checkpointThe standard (default) mode of ai_extract now runs on a new model checkpoint that the release notes describe as 2.3x faster, with better overall quality and no extra cost. Queries that already use the default mode pick up the change without edits.
ReleaseCatalogJDBC Unity Catalog connections are generally availableJDBC connections in Unity Catalog have reached general availability on serverless compute, Databricks Runtime 19 and above, and SQL warehouses 2025.40 and above, with SSL certificate support added. OAuth M2M authentication for these connections is still in Beta. Anyone who read the earlier Public Preview material should treat the GA scope above as the current one.

Thursday 1 October1 Oct 2026

PreviewData IngestionLakeflow Connect adds a managed GitLab connector in BetaLakeflow Connect now has a managed GitLab connector in Beta, ingesting repository, project, group, CI/CD and security data. It uses OAuth U2M authentication and supports incremental ingestion, UI and API pipeline authoring, and Declarative Automation Bundles. Workspace admins control access from the Previews page.
ReleaseCatalogCross-engine ABAC is generally available for external enginesSupported external engines can now read Unity Catalog managed tables while ABAC policies, row filters and column masks are enforced by Databricks serverless compute. The feature is generally available; requirements, supported clients and serverless compute costs are listed on the linked documentation page.
ReleaseLakebaseLTAP Direct Writes for Lakebase synced tables is generally availableLTAP Direct Writes, which writes the initial load and full refreshes of a Lakebase synced table straight to the storage layer instead of through the compute endpoint, has left Beta. You opt in per synced table under Synchronization settings, and it works on Lakebase projects running Postgres 16, 17 or 18.
ReleaseSQLBuilt-in IP and CIDR functions are generally available on Databricks Runtime 18 LTS+Databricks now ships native SQL, PySpark and Scala functions for IPv4 and IPv6 addresses and CIDR blocks: parsing, canonicalizing, inspecting, converting to binary, and ip_cidr_contains for containment joins. try_ variants return NULL on bad input instead of failing. They are generally available on Databricks Runtime 18 LTS and above.

Wednesday 30 September30 Sept 2026

ReleaseExcel and Google Sheets writeback now needs a workspace admin settingWriting data back to Databricks from the Excel add-in or from Google Sheets now requires a workspace admin to turn on the "Allow spreadsheet writeback" security setting. Workspaces where it stays off will see writeback stop working, so admins who have spreadsheet users should check the setting.
PreviewSQLai_decide enters Beta: fast structured decisions over text and data in SQLai_decide is a new AI Function in Beta that answers one or more questions about an input in a single call. For each question it returns a probability, one label from criteria you define, or a score on an ordered scale. The input can be plain text, a JSON object or array, or the VARIANT output of ai_parse_document or ai_extract. It runs from SQL for batch work and through a REST API for real-time use; the SQL function needs Databricks Runtime 15.4 LTS or above and is not available on Databricks SQL Classic. Typical uses are tagging, routing prompts to a model and judging agent answers.
ReleaseQuery HistoryQuery tags for SQL warehouses reach general availabilityQuery tags are now generally available. You attach custom key-value tags to SQL workloads to group, filter and attribute cost on Databricks SQL warehouses and Lakehouse//RT compute. The tags show up in system.query.history and on the Query History page.
ReleaseJobs & PipelinesTable update triggers in Lakeflow Jobs now cover OpenSharing objects and system tables (GA)Lakeflow Jobs table update triggers can now watch OpenSharing objects and system tables, and this is generally available. A job can start when a shared object or a system table changes, instead of on a schedule. Anyone polling these tables on a timer can consider switching to a trigger.

Tuesday 29 September29 Sept 2026

ReleaseGenieGenie One memory is generally availableGenie One can keep facts you give it, such as preferences and reporting conventions, and apply them in later conversations. Memories are private to each user and are not shared; you can ask Genie One to remember something or accept a suggestion it makes.
PreviewAgentsAgent Bricks CLI enters Beta for building and deploying custom code agentsThe Agent Bricks CLI (databricks-agentbricks) is a command-line tool in Beta for custom code agents. It scaffolds a project from a bundled framework template, runs it locally and deploys it to the Databricks agent runtime, with managed memory, sessions, tools and MLflow tracing set up from one authenticated command.
ReleaseUnity GatewayOpenAI GPT-6.1 Sol added to Unity Gateway, reachable through Foundation Model APIsGPT-6.1 Sol from OpenAI can now be called through Unity Gateway using Foundation Model APIs. The release note adds that customers remain responsible for complying with the applicable terms of the model provider.

Monday 28 September28 Sept 2026

ReleaseLakebaseLakebase Search gets lakebase_tokenizer for configurable keyword matchingA new lakebase_tokenizer extension controls how Lakebase Search splits text into terms: normalization and accent stripping, your own stopword and synonym lists, and stemming. It produces standard tsvector values, so it works with to_tsvector, GIN indexes and the lakebase_bm25 index; install it after enabling Lakebase Search in project settings.
ReleaseUnity GatewayxAI Grok 4.7 added to Unity Gateway as a pay-per-token Foundation ModelGrok 4.7 from xAI is now available on Unity Gateway and can be called through pay-per-token Foundation Model APIs. Customers remain responsible for complying with the applicable terms.
ReleaseLakebaseLakebase Search is GA: BM25 and vector search extensions in PostgresLakebase Postgres now ships two search extensions, lakebase_vector for approximate nearest-neighbour search and lakebase_text for BM25 full-text search, generally available on AWS and Azure. Together they allow semantic, keyword and hybrid search in one SQL query next to operational tables, without a separate search system. The vendor's own benchmarks compare it favourably with pgvector; the post also says Databricks AI Search remains the better fit for managed retrieval with little tuning.
PreviewData IngestionLakeflow Connect adds a 1Password Event Logs connector in BetaThe 1Password Event Logs connector is in Beta in Lakeflow Connect. It uses the 1Password Events API to ingest audit events, item usages and sign-in attempts, and authenticates with an Events Reporting bearer token. Workspace admins control access from the Previews page.
PreviewData IngestionLakeflow Connect adds an Akamai WAF connector in BetaThe Akamai WAF connector is in Beta in Lakeflow Connect and ingests web application firewall security events from Akamai. It authenticates with Akamai EdgeGrid API client credentials. Workspace admins control access from the Previews page.
PreviewData IngestionLakeflow Connect adds a CrowdStrike Falcon Event Stream connector in BetaThe CrowdStrike Falcon Event Stream connector is in Beta in Lakeflow Connect and ingests Falcon Event Stream events. It authenticates with OAuth 2.0 client credentials from a CrowdStrike API client that has Event streams Read permission. Workspace admins control access from the Previews page.
PreviewData IngestionLakeflow Connect adds a Workday Activity Logging connector in BetaThe Workday Activity Logging connector is in Beta in Lakeflow Connect. It incrementally ingests Workday user activity log events and authenticates with an OAuth refresh token. Workspace admins control access from the Previews page.
PreviewData IngestionLakeflow Connect adds a Microsoft 365 Unified Audit Logs connector in BetaA managed Microsoft 365 Unified Audit Logs connector is in Beta in Lakeflow Connect. It ingests unified audit events from Microsoft Entra ID, Exchange, SharePoint and other Microsoft 365 workloads, uses OAuth machine-to-machine authentication, and loads incrementally.
PreviewData IngestionLakeflow Connect adds an Okta System Logs connector in BetaA managed Okta System Logs connector is in Beta in Lakeflow Connect. It incrementally ingests audit and security events from the Okta System Log, authenticates with an SSWS API token, and supports Declarative Automation Bundles and API-based pipeline authoring.
DeprecatedThinking Machine Labs Inkling will retire on October 30, 2026

Sunday 27 September27 Sept 2026

PreviewUnity GatewayGateway usage table adds coding-agent session and prompt-cache token fieldsThe system.ai_gateway.usage system table now records coding-agent session metadata and the invocation source. New token-detail fields count tokens written to the 5-minute and 1-hour prompt caches, plus file and web search counts. The release notes label this Public Preview.

Saturday 26 September26 Sept 2026

ReleaseAppsHorizontal scaling for Databricks Apps is now GAA Databricks app can now run across multiple instances behind one app URL instead of just one, for higher availability and concurrency. Deployments get zero-downtime updates and best-effort session affinity between instances. This horizontal scaling capability has reached general availability.
CommunityCatalogHow to set up a governed JDBC connection in Unity CatalogWalks through Databricks' new Unity Catalog JDBC connection type (Public Preview on Databricks Runtime 18.1 and DBSQL 2025.40 and above): staging the driver JAR in a Unity Catalog volume, creating the connection as a securable object, and granting access with GRANT USE CONNECTION so callers never see the URL or credentials. Compares it against query federation and PySpark data sources so a reader can pick the right one for a given external source.

Friday 25 September25 Sept 2026

ReleaseJobs & PipelinesDeleting a Unity Catalog pipeline now keeps its tables unless you cascadeDeleting a Unity Catalog pipeline in the default publishing mode, from the UI or the Pipelines DELETE API, now leaves its materialized views, streaming tables and views in place. They can still be queried but cannot be updated until they move into an active pipeline. To remove the tables too, the API request must set the cascade field to true; Hive metastore pipelines are unaffected.
ReleaseGenieGenie Code can now build document processing pipelinesGenie Code can now orchestrate an end-to-end document processing pipeline, handling parsing, classification, extraction, and validation in one flow. Pointed at unstructured PDFs, contracts, or invoices, it checks extraction quality against labeled data and iterates to improve it.

Thursday 24 September24 Sept 2026

ReleaseCatalogTags for dashboards, notebooks, apps, and Genie Agents are GATags can now be applied to dashboards, notebooks, Databricks apps, and Genie Agents to organize workspace assets. The release is generally available and supports both governed and ungoverned key-value tags.
PreviewAI Searchai_search can query Lakebase synced tables, in BetaThe ai_search function can now retrieve from Lakebase synced tables that have Lakebase Search enabled, as well as from AI Search indexes. A call adds a lakebase_search knowledge source, and it can be combined with vector_search sources in the same call. The release notes label this Beta.

Wednesday 23 September23 Sept 2026

PreviewLakebaseLakebase writes its Postgres telemetry to system.lakebase tables (Beta)In Beta, Lakebase records query plans, session history, wait events, schema-change history, health data and Postgres logs into the system.lakebase system tables with no setup. Records are kept for 7 days, and account and metastore admins can read the tables and grant access to others.
DeprecatedDeepSeek V4 Pro (0813) will retire and be replaced by V4.1 Flash
PreviewCatalogTime travel can enforce ABAC policies on qualifying managed tablesIn Databricks Runtime 19, time travel can be used on qualifying Unity Catalog managed tables while ABAC row filter and column mask policies are enforced. The feature is in Beta, so learners should treat it as preview behaviour rather than generally available behaviour.
PreviewAppsGoverned agentic app building is in betaDatabricks says you can now build governed data and AI apps without code. Genie App Builder authors apps in natural language, App Spaces enforce governance, and Serverless Micro Apps scale to zero when idle.
ReleaseCatalogScala UDFs can now read Unity Catalog secretsSession-scoped and Unity Catalog Scala UDFs can now access secrets managed in Unity Catalog. Session-scoped UDFs use the caller's permissions, while Unity Catalog Scala UDFs declare secrets in a SECRETS clause and use the function owner's permissions. Users must check the stated environment, Databricks Runtime, access mode, and SQL warehouse requirements before using this in a UDF.
ReleasePython / PySparkUnity Catalog Python UDFs are GA for scalar and batch useScalar and batch Unity Catalog Python UDFs are now generally available on all supported compute types. The release adds named handlers for row-at-a-time scalar UDFs, removes the five-UDF-call-per-query limit, and aligns scalar TIMESTAMP inputs with PySpark behavior. Some capabilities require environment_version 6 or above, and requirements vary by feature and compute type.

Tuesday 22 September22 Sept 2026

ReleaseData IngestionVariant shredding will be added by default to new VARIANT tablesStarting November 18, 2026, Databricks will automatically add the variantShredding table feature to new tables with a VARIANT column when they are created on Databricks Runtime 17.3 or above. Variant shredding stores frequently accessed fields in a typed, columnar layout to improve query performance. Clusters on Databricks Runtime 15.4 cannot read from or write to those tables, so affected clusters should be upgraded to Databricks Runtime 16.4 or above before the change takes effect.
ReleaseWorkspaceCoding agents can be configured centrallyOrganizations can now centrally configure coding agents, including models, MCP servers, skills, a default harness and models, and Smart Routing. The settings apply when an agent starts, or can be rolled out through mobile device management.
ReleaseUnity GatewayUnified tracing includes coding-agent activityUnity Gateway adds coding-agent activity to unified tracing through the Unity Gateway CLI. Traces can show the tools and skills used alongside model requests in Govern > Traces or with Genie.
ReleaseWorkspaceug CLI connects coding agents to Unity GatewayThe Unity Gateway CLI (ug) can connect Claude Code, Codex, Pi, and other coding agents to Unity Gateway with one command. It handles authentication and configuration, adds approved MCP servers and skills, and shows spend in the terminal.
ReleaseCatalogUnity Catalog Pages added as a home for Genie Ontology knowledgeDatabricks describes Unity Catalog Pages as a governed home for business knowledge used with Genie Ontology. Readers working with Genie Ontology or Unity Catalog Pages should treat Pages as part of where business context for agents is managed.
ReleaseUnity GatewayOpenAI GPT-6 Luna is available on Unity GatewayOpenAI GPT-6 Luna is now available on Unity Gateway. It can be accessed through Foundation Model APIs pay-per-token, and customers are responsible for complying with applicable terms.
ReleaseUnity GatewayOpenAI GPT-6 Sol is available on Unity GatewayOpenAI GPT-6 Sol is now available on Unity Gateway. It can be accessed through Foundation Model APIs pay-per-token, and customers are responsible for meeting the applicable terms.
PreviewLakebaseLakebase backup schedules can now be managed through a Beta APIDatabricks added a Beta snapshot schedule API for Lakebase. It lets you read and set a root branch's automated snapshot cadences, including daily, weekly, or monthly schedules, and the retention period for each cadence.
PreviewData IngestionShopify connector is in Beta for Lakeflow ConnectLakeflow Connect now has a Shopify connector in Beta. It ingests Shopify e-commerce data, including products, orders, customers, inventory, fulfillments, and discounts, into Databricks, and workspace admins can manage access from the Previews page.

Monday 21 September21 Sept 2026

DeprecatedGenie One MCP beta endpoint is deprecated for a Unity Gateway service
ReleaseServingModel provider service API keys can use Unity Catalog secretsA model provider service API key can now be backed by a customer-owned Unity Catalog secret. The service reads the secret at query time, so rotating the secret takes effect without changing the service.
ReleaseQuery Historysystem.query.history is generally availableThe system.query.history system table is now generally available. It provides account-wide records for queries run on SQL warehouses or serverless compute, including execution status, duration metrics, and query source.

Friday 18 September18 Sept 2026

ReleaseLakebaseLakebase Search is generally availableLakebase Search is now generally available for Lakebase projects. It uses the lakebase_vector and lakebase_text Postgres extensions for vector search and BM25 full-text search in the same database, instead of using a separate search stack. Lakebase users who want this must enable Lakebase Search in project settings and install the extensions.
ReleaseAppsDatabricks Apps telemetry is generally availableConfiguring telemetry for Databricks Apps is now generally available. It collects traces, logs, and metrics using the OpenTelemetry protocol and stores them in Unity Catalog tables. After telemetry is enabled, Databricks captures system logs and usage events automatically, and app developers can add custom instrumentation with an OpenTelemetry SDK.

Thursday 17 September17 Sept 2026

PreviewComputeBase environments are in beta on classic computeBase environments can now be used to manage Python dependencies on classic compute. The beta supports both Databricks-provided environments and custom workspace-level environments.
DeprecatedMoonshot AI Kimi K2.7 will retire and Kimi K3 replaces it
PreviewCatalogABAC policies can now apply at the metastore level in BetaUnity Catalog now has Beta support for attaching an ABAC policy at the metastore level. A row filter, column mask, GRANT, or DENY policy attached there applies across every catalog in the metastore, including catalogs created later, instead of being replicated per catalog.
ReleaseGenieGenie One skills now support multiple files and foldersGenie One skills can now include multiple files and sub-folders. Each skill can use folders up to three levels deep and up to 50 files or folders, and skills can load text files only.

Wednesday 16 September16 Sept 2026

CommunityJobs & PipelinesA worked example of what Spark Declarative Pipelines solves, with F1 dataA practitioner walkthrough that builds a small pipeline over Formula 1 race data to show what Spark Declarative Pipelines (SDP) automates: resolving dependency order from declared datasets instead of hand-built DAGs, retries, state management and an event log built into the framework, and triggered runs that reuse Structured Streaming's micro-batch engine so the same declarative pipeline handles batch and streaming without separate code paths.
PreviewAgentsManaged agent memory and sessions are now in BetaDatabricks added Beta managed agent memory and managed agent sessions, two fully managed Lakebase-backed stores for agent state that work with agents built on any framework. Memory persists across conversations and supports semantic search; sessions store conversation history for the agent to read at the start of a turn and append to while it runs.
ReleaseCatalogABAC column masks can use VARIANT functions for ARRAY and MAP columnsABAC column masks can now use a single VARIANT function for ARRAY and MAP columns, in addition to STRUCT and other column types. This requires Databricks Runtime 19 or above, so readers using masks on ARRAY or MAP columns need to check their runtime before using it.
DeprecatedAnthropic models removed from AWS GovCloud IL5 workspaces
ReleaseWorkspaceUnused OAuth client secrets are deleted after 90 daysDatabricks now deletes a service principal's OAuth client secret after 90 days without use. If that deleted secret is still needed, create a new OAuth secret.
ReleaseUnity GatewayUnity Gateway API and developer tools are generally availableThe Unity Gateway API is generally available for managing model services, model provider services, and MCP services. Create, read, update, list, and delete operations are supported across Terraform, the Databricks CLI, and the listed Databricks SDKs; Declarative Automation Bundles support through the CLI is in Beta.

News

Top stories this week

Unity Catalog permissions on system.ai now control which AI Functions users can run (GA)

AI Search can index Streaming Tables, Materialized Views and managed Iceberg tables directly

Lakeflow Jobs show job-level query performance metrics and insights (GA)

Everything, day by day

Yesterday27 Oct 2026

Monday 26 October26 Oct 2026

Saturday 24 October24 Oct 2026

Friday 23 October23 Oct 2026

Thursday 22 October22 Oct 2026

Wednesday 21 October21 Oct 2026

Tuesday 20 October20 Oct 2026

Monday 19 October19 Oct 2026

Friday 16 October16 Oct 2026

Thursday 15 October15 Oct 2026

Wednesday 14 October14 Oct 2026

Tuesday 13 October13 Oct 2026

Monday 12 October12 Oct 2026

Friday 9 October9 Oct 2026

Thursday 8 October8 Oct 2026

Wednesday 7 October7 Oct 2026

Tuesday 6 October6 Oct 2026

Monday 5 October5 Oct 2026

Friday 2 October2 Oct 2026

Thursday 1 October1 Oct 2026

Wednesday 30 September30 Sept 2026

Tuesday 29 September29 Sept 2026

Monday 28 September28 Sept 2026

Sunday 27 September27 Sept 2026

Saturday 26 September26 Sept 2026

Friday 25 September25 Sept 2026

Thursday 24 September24 Sept 2026

Wednesday 23 September23 Sept 2026

Tuesday 22 September22 Sept 2026

Monday 21 September21 Sept 2026

Friday 18 September18 Sept 2026

Thursday 17 September17 Sept 2026

Wednesday 16 September16 Sept 2026

The same news, without the checking