Laya vs zero-shot NLI classifiers: which fits your task?

Zero-shot NLI classification uses a natural language inference model to test each candidate label as a hypothesis, such as "This text is about billing", and ranks labels by how strongly the text supports each one. It needs no training data but runs one pass per label. Laya scores all options together in one pass.

Natural language inference (NLI) models are trained on premise–hypothesis pairs and predict entailment, neutral or contradiction. Yin, Hay and Roth (2019) observed that this gives a classifier for free. Take the input text as the premise, turn each candidate label into a hypothesis such as "This text is about billing.", and rank labels by entailment probability. The facebook/bart-large-mnli checkpoint on the Hugging Face Hub, served through the zero-shot-classification pipeline, made the method a default for teams who needed classification without training data.

The method has three properties you will notice in production:

1. One forward pass per label. Ten labels means ten premise–hypothesis pairs. Batching keeps it on the GPU, but compute grows linearly with the label count.
2. Labels are judged independently. The model never sees "billing" and "refund" side by side. It scores each against the text and the scores are normalised afterwards.
3. Template sensitivity. "This example is {label}." and "The customer wants {label}." can give noticeably different results, and there is no training signal telling you which to use.

Laya Studio is an independent API powered by the open-source Laya model. It is not affiliated with Laya's authors, Convai Innovations. For the concepts, see zero-shot vs fine-tuned and typed decisions.

Laya is also an encoder (ModernBERT-large for English, mmBERT-base for 100+ languages), but it was trained for the decision task directly rather than on NLI pairs. For each question it builds a single sequence containing the instructions, every option and the state:

[CLS] choice question: Which team should handle `body`? [SEP]
  [MASK] billing: invoices, payments, refunds
  [MASK] technical: bugs and outages
  [MASK] sales: pricing [SEP]
  {"body": "..."} [SEP]

A decision head reads one logit at each [MASK], and the softmax over them is the answer. This differs from NLI in two ways:

• The options see each other. Self-attention runs across all the option texts, so the model can tell "refund" apart from "billing_question" because both are in view. That matters for fine-grained, overlapping labels.
• One sequence per question, and every question in the same pass. A request with a 5-way choice, a 4-level score and three yes/no flags is scored in one forward pass. NLI would need 5 + 4 + 3 = 12 premise–hypothesis evaluations for the same request, plus glue code to interpret the score as ordinal.

The response is typed. choice returns the winning key, the per-option probabilities and a confidence. score returns the probability-weighted level and a legend. noul returns P(true). See choice, score and noul and option-marker scoring.

The usual NLI pipeline in Python:

from transformers import pipeline

clf = pipeline("zero-shot-classification", model="facebook/bart-large-mnli")
text = "We were billed twice for March. Please refund the duplicate today or we will cancel."
out = clf(text, candidate_labels=["billing", "technical", "sales"],
          hypothesis_template="This request is about {}.")
print(out["labels"][0], out["scores"][0])

# urgency and churn risk need separate calls and your own interpretation
urgent = clf(text, candidate_labels=["urgent"], multi_label=True)

The same decisions as one Laya Studio request:

curl -s https://api.laya.studio/v1/systemone \
  -H "Authorization: Bearer lsk_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "state": {"body": "We were billed twice for March. Please refund the duplicate today or we will cancel."},
    "questions": {
      "department": {"type": "choice", "instructions": "Which team should handle `body`?",
        "criteria": {"billing": "invoices, payments, refunds", "technical": "bugs and outages", "sales": "pricing"}},
      "urgency": {"type": "score", "instructions": "How urgent is the request in `body`?",
        "criteria": ["no time pressure", "needs attention soon", "blocking issue or hard deadline"]},
      "churn_risk": {"type": "noul", "instructions": "Does the sender threaten to cancel or leave?"}
    }
  }'

The response carries answers keyed by your question ids, a usage block with input_tokens and output_tokens: 0, and a routing object that says which checkpoint answered and why. Laya picks the English or the multilingual checkpoint from the script and language of the state. The same call from TypeScript:

const r = await fetch('https://api.laya.studio/v1/systemone', {
  method: 'POST',
  headers: { Authorization: `Bearer ${key}`, 'Content-Type': 'application/json' },
  body: JSON.stringify({ state, questions }),
});
const { answers, routing } = await r.json();
console.log(answers.department.choice, answers.urgency.score, answers.churn_risk.noul, routing.model);

The NLI approach costs one encoder pass per candidate label, per question. For a single question with a few labels on a GPU that is fast. The cost grows with every label and every extra question, and on CPU it becomes the bottleneck quickly.

Laya's figures from its model card (Tesla T4, in-process, excluding network):

We do not quote NLI latency numbers here because they depend heavily on model size, label count, hardware and batching. Measure both on your own label set. The structural point holds either way: NLI work scales with questions × labels, while Laya's scales with questions, and those share a batch.

On cost, self-hosted NLI is free apart from hardware, and so is self-hosted Laya, since its weights are Apache-2.0. Laya Studio charges per input token (1 credit = 1 input token), 30% below Jev's list price, with 5 free runs; see pricing.

Laya's own model card reports results on XNLI, the cross-lingual NLI benchmark that zero-shot NLI models are usually trained or evaluated on:

On topic-style classification the card reports AG News (4 labels) at 0.950 routed, and DAIR Emotion (6 labels) at 0.595. On MASSIVE intent with 20 options, the routed system scores 0.783 on English and 0.451 across 13 other languages. The multilingual checkpoint clears three times random on 45 of the 51 MASSIVE languages.

For non-English work with NLI you need a multilingual NLI model, typically an XLM-R model trained on XNLI. You also have to choose it yourself. Laya Studio routes by script before inference. That matters because the English checkpoint does not degrade gracefully outside English: on Khmer it scored 0.000 accuracy at 0.952 confidence. See language routing and multilingual classification.

Where NLI can hold up better: large label sets. Each NLI hypothesis gets the model's full attention, while Laya's options share one head_max_len budget (192 tokens on English, 256 on multilingual). At 77 labels, Laya's Banking77 accuracy is 0.425. For big taxonomies, split them into a coarse-to-fine hierarchy with Laya, or keep a per-label scorer.

Where both are weak: novel multi-field business rubrics with no fine-tuning. Laya's base checkpoints score 0.362 on the typed-decisions benchmark, below the majority-class baseline. A fine-tuned Laya reached 0.766.

NLI zero-shot scores are entailment probabilities re-normalised across your labels. They were never trained to be calibrated class probabilities for your task, and in multi-label mode each label gets an independent entailment score whose scale depends on the template. Teams usually end up picking thresholds by trial and error.

Laya is trained with RLCD, reinforcement learning whose reward is a strictly proper scoring rule: log score plus spherical score, and a ranked probability score for ordinal questions. The intent is that honest probabilities maximise reward (see proper scoring rules). The model card is candid that the checkpoints still ship over-confident. Refitting one temperature per (question type, option count) on held-out data moves mean ECE from 0.466 to 0.081 on the English checkpoint and from 0.314 to 0.106 on the multilingual one.

In practice:

• Fit temperatures on your own labelled data before setting act/escalate thresholds. See temperature scaling and calibrated probabilities.
• Threshold on confidence (the normalised entropy 1 - H(p)/log(k)), not on action.act_probability. The model card reports that the act head reads about 1.0 for almost every input and carries no usable signal yet.
• For ordinal rubrics, remember that score is Laya's weakest primitive (SST-5 0.372). Validate before automating on it.

Stay with an NLI zero-shot classifier when:

• You have one question with a handful of labels and low volume.
• You already self-host it and the accuracy is acceptable.
• Your labels are numerous but each is easy to phrase as a clean hypothesis, and you can afford one pass per label.

Move to Laya when:

• You ask several questions about each input (a category, an urgency level, a few yes/no flags) and want them in one pass with typed outputs.
• You need ordinal scores, not just labels.
• Latency matters, or label count and question count are growing.
• Inputs arrive in many languages and you do not want to manage a separate multilingual model.
• You want a path to fine-tune the same model for a specific workflow.

Keep in mind the limits covered above: under ~20 options per choice question, calibration before thresholds, and weak zero-shot performance on novel multi-field rubrics. To compare on your own data, create a free account, send the labels you use with NLI today, and read the docs for the full question schema. Related comparisons: Laya vs fine-tuned BERT and Laya vs embeddings + kNN.