Salesforce's Koa Decisions Routes Requests in 26 Milliseconds. An LLM Takes 1.4 Seconds.

Salesforce’s Koa Decisions, the next version of its HyperClassifier routing model, took about 26 milliseconds per routing decision in Salesforce’s measurements, against about 1.4 seconds for a general-purpose large language model (LLM).

Routing is the step where an agent decides which of its parts handles a customer’s message. Salesforce built Koa Decisions to make that call without a larger, more expensive model, which leaves the LLM for writing and reasoning.

How does it decide?

Koa Decisions reads a request and a list of categories a team defines, then returns one choice with a confidence score for each option. A customer who types “I can’t log in to my account” lands with accounts rather than engineering or sales. Salesforce’s data science team built it by applying new training techniques to Gemma, Google’s open-weight model, and added support for images and audio.

An LLM writes its answer one token, the basic unit of text a model processes, at a time. A decision model reads the whole request at once and returns a short answer in one step. That is also why the next question is how a team changes the options.

What changes when a category is added?

The old way was a custom machine learning model. A team defined the outcome, gathered and labeled examples, trained the model, checked its accuracy and repeated the cycle. Salesforce says that took months for many enterprise customers.

Koa Decisions starts from a language model that already knows how “send back the order” relates to “returns.” Salesforce says a team can add a category and sometimes reach production in hours. In its demo, adding “Returns” to a list of three options sent a returned-order request there at once, with no retraining.

What can a buyer try, and what is planned?

Salesforce is introducing Koa Decisions this month, and anyone can run the demo through a free Trailhead account, customer or not. Salesforce plans to extend it into flows, Apex and prompt templates for builders, plus an API for developers, and gave no dates. It did not say what a routing decision costs.

Salesforce’s own example splits the work: a decision model flags a slipping deal and picks the team to step in, and an LLM summarizes what changed. Which steps in a company’s own agents pick from a list of options rather than write text?