NEAREST BY: a SQL join for batch vector search in Databricks Runtime

NEAREST BY is a SQL join that finds, for each row on the left, the k closest rows on the right by vector similarity or distance, either EXACT or APPROX. It runs in Photon with a fused operator, and APPROX queries can use an IVF index stored as a liquid-clustered Delta table. The post does not state a release stage, so availability needs checking against the documentation.

Read the original