MLflow review queues
A reviewer workflow for labelling traces one item at a time, for people who are not workspace users.
Why it is worth watching
Expert feedback is the input that makes an LLM judge worth trusting.
What Beta promises
Open to most customers but not for production: support runs through engineering, and the shape of it can still move. On by default on Premium, off on Enterprise.
| Who can use it | Most customers |
|---|---|
| Production use | No |
| Support | Engineering only |
| On by default | Premium yes, Enterprise no |
The definitions are Databricks' own, on its release types page. The label on this page is the one the feature's documentation states, last checked on 13 Sept 2026.
Where to read more
It is part of something we wrote up: Human feedback on generative AI output — Human judgement reaches MLflow as assessments on a trace, from developers annotating in the UI, from experts working a review queue, and from end users pressing thumbs up or down.