Laya vs embeddings + kNN: which text classifier fits your task?

Embeddings + kNN labels new text by turning it into a vector and copying the label of its nearest stored examples. Laya is a decision model that reads the text together with your question and options, and scores each option directly with a probability. Embeddings compare by similarity; Laya decides with the question in view.

An embedding classifier has two parts. An encoder maps text to a fixed vector, and a rule turns vectors into labels. The common rules are:

• k-nearest neighbours (kNN): embed a labelled training set once, and at query time find the k closest examples and take a (possibly weighted) vote.
• Nearest centroid: average the example vectors per label and pick the closest average.
• Label-description matching: embed a sentence describing each label, and pick the label whose description is most similar to the input. This is zero-shot, but usually the weakest variant.

Sentence-embedding models such as Sentence-BERT made this practical. Vector libraries such as FAISS make it scale to millions of vectors.

Laya is a different kind of model, a cross-encoder-style decision model. The state, the question's instructions and every option go into one sequence. Each option sits behind its own [MASK] marker, and a bidirectional encoder (ModernBERT-large or mmBERT-base) reads everything together. A small head scores each marker, and a softmax gives the answer distribution. The label set is part of the input, so it can change on every request. See option-marker scoring.

The core difference: an embedding compresses the input before it knows the question. Laya reads the input with the question and options in view.

Be clear about when you should not use Laya.

• Very large label spaces. A vector index handles 5,000 product categories without trouble. Laya's options share a fixed budget of 192 tokens (English) or 256 tokens (multilingual). The model card shows the cost: on Banking77, with 77 labels, Laya scores 0.425, because each label gets about 3–4 tokens. The card's advice is to keep choice questions under about 20 options.
• Plenty of labelled history. If you already have 50,000 tickets labelled by your team, kNN uses them directly and reflects your own labelling conventions, including the idiosyncratic ones. Laya uses your labels only if you fine-tune it.
• Lowest cost per item. Embedding one short text and querying an index is a single encoder pass plus a lookup, and the example vectors are computed once. Laya encodes the state once per question, because each question is its own sequence in the batch.
• Deduplication, search and clustering. These are similarity problems, not decisions, and embeddings are the right primitive for them.
• Evidence you can show. "This ticket looks like these five past tickets" is an explanation many reviewers find persuasive.

• No labelled data yet. You write the options as descriptions ("billing": "invoices, payments, refunds") and get answers on day one. kNN with no examples falls back to label-description matching.
• Instructions matter. The same email can be asked "which team should handle this?" and "is the sender threatening to cancel?". An embedding of the email is the same vector for both. Laya conditions each answer on the question.
• Typed answers other than a label. Laya's score returns an expected value over ordered levels plus a distribution, and noul returns the probability that a statement holds. Doing these with kNN means building a separate regressor or a binary index per question.
• Negation and fine conditions. "I do not want a refund, just an explanation" sits close to refund requests in many embedding spaces. A model that reads the option text against the full input has a better chance of catching the negation. It is still not guaranteed, so test it on your own data.
• Probabilities you can act on. kNN vote fractions (3 of 5 neighbours) are coarse and uncalibrated. Laya is trained against strictly proper scoring rules, so its outputs are meant to be probabilities. After refitting temperatures on held-out data, the model card reports mean ECE falling from 0.466 to 0.081. See calibrated probabilities.
• Many questions in one call. Laya answers all questions in one request, and batching makes each additional question cheaper: 10 questions take 72.3 ms on the multilingual checkpoint, against 32.8 ms for one.

With kNN at k = 5, a label that wins 4 of 5 votes is often reported as "0.8 confidence". Several things are wrong with that number:

1. It can take only six values (0, 0.2, 0.4, 0.6, 0.8, 1.0).
2. It depends on k and on how dense each class is in the training set. A large class wins more votes simply by being large.
3. Nothing trained it to match observed accuracy.

Cosine similarity has the same problem. A score of 0.82 means different things in different embedding models and for different labels.

You can calibrate kNN, for example with Platt scaling or isotonic regression on held-out predictions. Many teams skip that step. Laya's training objective optimises calibration directly: the reward is log score plus spherical score (and ranked probability score for ordinal questions). The model is still shipped over-confident, and the Laya authors say so. Fitting one temperature per (question type, option-count bucket) is the fix, and it takes minutes on a few hundred labelled answers. See temperature scaling and expected calibration error.

The laya package ships this pattern as predict_shortlist in laya.shortlist. For each choice question with more than k options (default 20), it:

1. embeds the state and every option with an embed_fn you supply;
2. keeps the top k options by cosine similarity;
3. runs one Laya forward pass over the reduced option set.

import laya
from laya.shortlist import predict_shortlist
from sentence_transformers import SentenceTransformer

agent = laya.load("convaiinnovations/laya")
encoder = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

def embed_fn(texts):
    return encoder.encode(list(texts), normalize_embeddings=True)

intents = {name: desc for name, desc in load_banking_intents()}   # 77 labels
questions = {
    "intent": {"type": "choice",
               "instructions": "What does the customer want?",
               "criteria": intents},
    "urgent": {"type": "noul", "instructions": "Is the customer blocked right now?"},
}

res = predict_shortlist(agent, {"message": "My card payment was declined twice today"},
                        questions, embed_fn, k=20)
print(res["answers"]["intent"]["choice"], res["shortlist"]["intent"]["labels"][:5])

Non-choice questions pass through unchanged. A choice question with k or fewer labels skips the embedding step. The probabilities on a shortlisted question cover the kept labels only. The package docstring is careful about evidence: the coarse-to-fine pattern comes from a community report (issue #102), and "this module does not measure" the Banking77 figures in that report. Measure the shortlist's recall on your own labels before you trust it. If the right label is not in the top 20, Laya cannot pick it.

If you only have the Laya checkpoint in memory, embed_fn_from_agent(agent) mean-pools Laya's own encoder. The docstring notes that "a dedicated bi-encoder passed as embed_fn will usually shortlist better".

With Laya Studio

Do the shortlist on your side and send only the top 20 labels:

const top = await shortlist(message, allIntents, 20);   // your vector index
const res = await fetch("https://api.laya.studio/v1/systemone", {
  method: "POST",
  headers: { Authorization: `Bearer ${process.env.LAYA_API_KEY}`, "Content-Type": "application/json" },
  body: JSON.stringify({
    state: { message },
    questions: {
      intent: { type: "choice", instructions: "What does the customer want?",
                criteria: Object.fromEntries(top.map((t) => [t.name, t.description])) },
    },
  }),
}).then((r) => r.json());

That is billed per input token read, once per question. The embedding lookup runs in your own infrastructure.

For a fixed label set and heavy traffic, embeddings are hard to beat on cost. You embed each input once, and index lookups are cheap. Laya's cost scales with the number of questions, since each question is a separate row in one batched forward pass:

These are T4 GPU timings from the Laya model card, measured in-process. Over HTTP to Laya Studio, add your network round trip. Billing is per input token (1 credit = 1 input token), 30% below Jev's list price, with 5 free runs; see /pricing.

Rules of thumb:

• If you ask one question over a fixed set of hundreds of labels at very high volume, use embeddings.
• If you ask several different questions per item, such as a team, an urgency score and three yes/no flags, Laya answers all of them in one call. You would otherwise build and maintain several separate kNN setups.
• If you need both, shortlist then decide. The embedding step bounds the label count, and Laya handles the final call and the typed questions.

Embeddings + kNN

• Quality depends on the training set. Mislabelled or stale examples are copied into predictions.
• There is no instruction channel, so each question needs its own index, examples or rules.
• Similarity is not entailment. Topically close texts can need different labels.
• Scores are not probabilities without calibration.

Laya

• Large label spaces need a shortlist or a hierarchy (Banking77: 0.425 at 77 labels).
• Context per question is 512 tokens (English) or 1,024 (multilingual), so long states are truncated.
• Base checkpoints are near chance on the complex typed-decisions workflows zero-shot (0.362). That benchmark needed fine-tuning to reach 0.766.
• score questions are the weakest primitive (SST-5 0.372). action.act_probability carries no usable signal yet.
• The model ships over-confident. Refit temperatures before you gate on probabilities.

Try both on a sample of your own traffic. A new Laya Studio workspace gets 5 free runs; after that a test set costs its input tokens at a list price 30% below Jev's. The request format is in the docs.