Product area
Serving
Model Serving: endpoints for custom models, foundation models, and agents; traffic, scaling, monitoring.
Concepts
- 1Model serving endpointsA serving endpoint is a managed REST API in front of one or more models: a custom model version from models in uc packaged with MLflow, or a foundation model, either Databricks-hosted or from an external provider such as OpenAI. Databricks runs the serverless…Intermediate
- 2Foundation Model APIsFoundation Model APIs are Databricks-hosted model serving endpoints for a curated set of chat and embedding models, reachable the moment your workspace is created — no deployment step, no GPU to provision. They come in two billing modes, pay-per-token and…Intermediate
- 3Paying for a foundation model: tokens, units and reservationsA Databricks-hosted foundation model is billed in one of four ways, and the choice is a capacity decision rather than a model decision: the same model can usually sit behind more than one of them. Pay-per-token draws on a shared pool and charges per input and…Intermediate
- 4Batch inference with ai_queryaiquery calls a model from SQL. Point it at an endpoint, give it a prompt built from your columns, and it returns the model's answer as a column. Run it over a table and you have batch inference, with no loop, no notebook and no serving client.Intermediate
- 5Structured outputsStructured outputs are a responseformat field on a chat request that tells the serving layer what shape the answer must take. It works with any supported chat model on Foundation Model APIs, both pay-per-token and provisioned throughput, and you send the same…Intermediate
- 6Serving compute and scalingEvery custom model serving endpoint has four settings that decide what it costs and how it behaves under load: the hardware it runs on (workloadtype), how much of that hardware it gets (workloadsize, or minprovisionedconcurrency and…Advanced
- 7External models and model provider servicesDatabricks has two ways to put a model it does not host behind a Databricks address.Intermediate
Also covered here
News
Resources
Priority pay-per-token for Foundation Model APIsDocumentation · Databricks · checked 24 Oct 2026Free
Nothing of that kind for this area. Try the full list.