Intent detection with a calibrated decision model

Intent detection is working out what a user wants from what they typed or said, and mapping it to one of the actions your product supports. Laya takes the message and your list of intents, returns a probability for each in one fast call, and lets you confirm, clarify or hand off based on how confident it is.

Intent classification is the first decision in almost every conversational system: a chatbot, a voice IVR, an in-app assistant, a search box that behaves differently for "cancel" than for "pricing". It looks solved. Benchmarks like Banking77 and MASSIVE have been around for years, and a fine-tuned BERT on a few thousand labelled utterances gets strong accuracy on a stable label set.

The trouble in production is that the label set is not stable. Product adds a feature and needs a new intent. Legal wants "report fraud" split from "dispute a charge". A new market launches in another language. Each change to a fine-tuned classifier means collecting examples, relabelling, retraining, re-evaluating and redeploying, and in the meantime the new intent falls into "other".

The alternative most teams reach for is prompting a large language model with the list of intents. That adapts instantly but brings its own costs: hundreds of milliseconds to seconds per turn in a conversation where the user is waiting, a per-token bill on every message, output that must be parsed back into a label, and a confidence signal that is at best a self-reported number the model generated as text.

A decision model sits between the two. With Laya the intent list is part of the request: each label and its description are rendered as options, each option is scored at its own option marker, and a softmax over those scores gives you a distribution. Changing the intent set is a code change, not a training run.

In a conversation, intent detection sits on the critical path of every turn, so latency is the first constraint. Laya is non-autoregressive: it reads the conversation and all options together in one pass of a bidirectional encoder and does not generate tokens. The model card reports 39.5 ms per single-question call on the English checkpoint and 32.8 ms on the multilingual checkpoint, both on a T4 GPU; three questions on the same turn share that pass. Over the hosted API, add your network round trip. That leaves room in a voice or chat latency budget for the actual response.

The second constraint is what you do with the answer. A good intent layer does not just pick a label; it decides between acting, confirming ("Do you want to update your address?") and clarifying ("Is this about your card or your address?"). That decision needs a probability you can trust. Laya is trained with RLCD against proper scoring rules, and the response gives you the full distribution plus a confidence score, so "the top two intents are close" is visible in the numbers rather than hidden behind a single label.

Third, cost. Laya Studio charges per input token (1 credit = 1 input token), 30% below Jev's list price; a three-question intent call reads the turn three times, so short turns are cheap and long transcripts cost more. See pricing.

Finally, there is no text to parse and no invented intent. The model can only return a key you sent. See the comparison with zero-shot NLI and embedding k-NN for other request-time approaches.

State. Pass the conversation as a list of turns. Laya accepts text, JSON objects and turn lists; the example sends the last assistant turn and the user turn so that short replies such as "yes, that one" can be interpreted in context. Point the instruction at "the last user turn" so the model knows which part to classify.

Labels. Use a choice question with keys that are your internal intent ids and values that describe the intent in plain language. The description is what the model reads, so "change a postal or billing address on the account" beats "addr_upd". Include an explicit out_of_scope option and a talk_to_human option; users ask for both constantly and both need special handling.

Cardinality is the main design constraint. All options share a fixed token budget for the question head (head_max_len: 192 tokens on the English checkpoint, 256 on multilingual). With 8 options each gets plenty of room. With 77, as in Banking77, each gets roughly 3 to 4 tokens and labels become indistinguishable. The model card is candid about this: Laya scores 0.425 on Banking77 against 0.870 published for TypeSafe Jev, while on 4-label AG News it scores 0.950 against 0.910. Keep a single question to about 20 options or fewer. For a large taxonomy, use two steps: a coarse question over 6 to 10 domains, then a fine question over the intents in the chosen domain. The laya package also ships an opt-in embedding shortlist (predict_shortlist) that narrows a large label set with your own embedding function before the decision call.

Extra flags. secondary_intent catches the "and also" utterances that single-label classification drops. needs_clarification gives you a direct signal for vague input instead of inferring it from low confidence alone. Both are noul questions, which the model card notes can follow their false/true labels on the English checkpoint (issue #156); if one looks stuck, rephrase it as a two-option choice with neutral keys.

A three-band policy on intent.confidence works well for conversational flows:

Two details matter. First, for choice questions confidence is 1 - H(p) / log(k), one minus normalised entropy, not the top probability. With eight options, a top probability of 0.6 can come with a confidence well under 0.6 if the rest is spread across several intents. Use probabilities to build the clarification prompt from the top two keys.

Second, action.act_probability exists in every answer but the model card states it carries no usable signal yet (issue #185): it reads close to 1.0 for nearly all inputs, and on 396 labelled decisions its raw logits scored an AUROC of 0.30 against 0.77 for confidence. Log it, but do not let it decide whether the bot acts.

The bands above are illustrations. Out of the box the checkpoints are over-confident; the card reports that refitting a temperature per question type and option count cut mean ECE from 0.466 to 0.081 on the English checkpoint. Take a few hundred real transcripts, label the intents, and choose bands from a reliability diagram of your own traffic. See act and escalate routing for the general pattern.

curl -s https://api.laya.studio/v1/systemone \
  -H "Authorization: Bearer lsk_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "state": [{"role": "user", "content": "my card still has the old address, can you fix it"}],
    "questions": {
      "intent": {"type": "choice",
        "instructions": "What does the user want in the last user turn?",
        "criteria": {"update_address": "change a postal or billing address on the account",
                     "card_replacement": "order a new or replacement card",
                     "talk_to_human": "explicitly asks for a person or agent",
                     "out_of_scope": "anything the assistant cannot help with"}}
    }
  }'

Python:

import os
import requests

API = "https://api.laya.studio/v1/systemone"
HEADERS = {"Authorization": f"Bearer {os.environ['LAYA_API_KEY']}"}

def detect(turns: list, questions: dict) -> tuple[str, str | list[str]]:
    r = requests.post(API, headers=HEADERS,
                      json={"state": turns, "questions": questions}, timeout=3)
    r.raise_for_status()
    a = r.json()["answers"]
    intent = a["intent"]
    if intent["choice"] == "talk_to_human":
        return "handoff", "talk_to_human"
    if intent["confidence"] >= 0.6:
        return "act", intent["choice"]
    top2 = sorted(intent["probabilities"], key=intent["probabilities"].get, reverse=True)[:2]
    if intent["confidence"] >= 0.35:
        return "confirm", intent["choice"]
    return "clarify", top2

TypeScript:

type ChoiceAnswer = {
  type: 'choice';
  choice: string;
  probabilities: Record<string, number>;
  confidence: number;
  action: { act_probability: number };
};

export async function detectIntent(turns: { role: string; content: string }[], questions: object) {
  const res = await fetch('https://api.laya.studio/v1/systemone', {
    method: 'POST',
    headers: { Authorization: `Bearer ${process.env.LAYA_API_KEY}`, 'Content-Type': 'application/json' },
    body: JSON.stringify({ state: turns, questions }),
  });
  if (!res.ok) throw new Error(`laya ${res.status}`);
  const { answers } = await res.json();
  const intent = answers.intent as ChoiceAnswer;
  const ranked = Object.entries(intent.probabilities).sort((x, y) => y[1] - x[1]);
  return { intent: intent.choice, confidence: intent.confidence, top2: ranked.slice(0, 2) };
}

Sign up at /signup for a key; the docs describe the full response.

• High-cardinality label sets. Above roughly 20 options accuracy drops sharply because options share a fixed token budget; Banking77 at 77 labels is 0.425. Use a coarse-to-fine hierarchy or a shortlist.
• Zero-shot is not free accuracy. English MASSIVE intent is 0.783 on the English checkpoint per the model card: useful, but short of what a model fine-tuned on your own intents can reach. If you have labelled data and a stable taxonomy, compare against a fine-tuned BERT.
• Non-English utterances. The English checkpoint falls to near chance on non-Latin scripts while staying confident. Let the router choose the checkpoint, or read multilingual intake.
• Context. Only the recent turns fit. Summarise long sessions.
• act_probability is not yet a reliable act/abstain signal; gate on confidence.