I tested Databricks AI_DECIDE() against 20 LLMs

AI_DECIDE is currently free in beta, so I tried it on a fairly simple task: deciding whether to approve or reject a post. Managed inference is free, but running the notebook still costs compute.

To compare accuracy, speed, and cost, I put it against the 20 models available via the Databricks AI Gateway (for example, Llama, Qwen, DeepSeek, GPT-6, Claude, Gemini, Grok, and OpenJev).

You can access the article on the SunnyData blog for free.

What is AI_DECIDE()?

AI_DECIDE takes natural-language instructions (text or structured data) and returns a result you can use directly in SQL. You can also use the REST API. The result can be one of your labels, a probability between 0 and 1, or a score on a scale that you define.

You can use it to categorize a travel expense, check a hotel’s pet policy, or score a guest review. In this case, I’m using its choice output for moderation (approved or refused).

What did I test?

I limited the sample to five posts: a standard SQL question, a post criticizing a product release, a money scam, a direct threat, and a safety report that contained a threat. The expected decisions were approve, approve, refuse, refuse, and approve so that the test could compare model…