Build your own Jev (100% local)

Everything you need to turn an open-source LLM into a fast, local decision engine without retraining it. It covers next-token scoring, fixed choices with probability distributions, SGLang, and a hands-on benchmark against standard text generation on the same model.

Many LLM calls do not need newly written text. The application already knows the possible answers, and it only needs the model to choose one.

Suppose a support ticket says, “I was charged twice for the same subscription.”

The application needs to send it to one of three teams: billing, technical support, or account access.

A normal LLM call asks the model to write an answer. It might return a sentence, a label, or a JSON object. The application waits for that text, parses it, and extracts the selected team.

That is unnecessary if every valid answer is already known.

In fact, a better and more efficient way to do this is to treat the same request as a decision (what Jev does). The application provides the ticket/query and the three allowed answers to Jev. The model then returns a score for each answer in one go (we’ll discuss shortly how exactly you can do that):

billing             0.91
technical support   0.06
account access      0.03