Product area

Serving

Model Serving: endpoints for custom models, foundation models, and agents; traffic, scaling, monitoring.

GenAI Engineer ML Associate Machine Learning Generative AI

Concepts

6 in this area
  1. 1Model serving endpointsA serving endpoint is a managed REST API in front of one or more models: a custom model version from models in uc packaged with MLflow, or a foundation model, either Databricks-hosted or from an external provider such as OpenAI. Databricks runs the serverless…
  2. 2Batch inference with ai_queryaiquery calls a model from SQL. Point it at an endpoint, give it a prompt built from your columns, and it returns the model's answer as a column. Run it over a table and you have batch inference, with no loop, no notebook and no serving client.
  3. 3Structured outputsStructured outputs are a responseformat field on a chat request that tells the serving layer what shape the answer must take. It works with any supported chat model on Foundation Model APIs, both pay-per-token and provisioned throughput, and you send the same…
  4. 4Serving compute and scalingEvery custom model serving endpoint has four settings that decide what it costs and how it behaves under load: the hardware it runs on (workloadtype), how much of that hardware it gets (workloadsize, or minprovisionedconcurrency and…
  5. 5Paying for a foundation model: tokens, units and reservationsA Databricks-hosted foundation model is billed in one of four ways, and the choice is a capacity decision rather than a model decision: the same model can usually sit behind more than one of them. Pay-per-token draws on a shared pool and charges per input and…
  6. 6External models and model provider servicesDatabricks has two ways to put a model it does not host behind a Databricks address.

News

No news for this area.

Resources

Use ai_queryDocumentation · DatabricksFreeTransform unstructured data using AI FunctionsDocumentation · DatabricksFreeExternal models in Model ServingDocumentation · DatabricksFreeGovern external model providers (model provider services)Documentation · DatabricksFreeCreate and manage external model providers (model provider services)Documentation · DatabricksFreeModel Provider Service | Databricks API ReferenceDocumentation · DatabricksFreeAnalyze Unity Gateway costDocumentation · DatabricksFreeRoute optimization on serving endpointsDocumentation · DatabricksFreeOn-demand provisioned throughput Foundation Model APIsDocumentation · DatabricksFreeModel units in provisioned throughputDocumentation · DatabricksFreeReserved provisioned throughput Foundation Model APIsDocumentation · DatabricksFreePriority pay-per-token for Foundation Model APIsDocumentation · DatabricksFreeFoundation Model APIs limits and quotasDocumentation · DatabricksFreeCustom models overviewDocumentation · DatabricksFreeOptimize Model Serving endpoints for productionDocumentation · DatabricksFreeModel Serving limits and regionsDocumentation · DatabricksFreeModel Serving conceptsDocumentation · DatabricksFreeStructured outputs on DatabricksDocumentation · DatabricksFreeFoundation model REST API referenceDocumentation · DatabricksFreeDeploy models using Model ServingDocumentation · Databricks · checked 10 Sept 2026FreeCreate custom model serving endpointsDocumentation · Databricks · checked 10 Sept 2026FreeInference tables for monitoring and debugging modelsDocumentation · Databricks · checked 10 Sept 2026FreeAI governance with Unity GatewayDocumentation · Databricks · checked 10 Sept 2026Freeai_query functionDocumentation · Databricks · checked 10 Sept 2026Free