Laya vs GPT and Claude: which should classify your text?

An LLM classifier is a general model such as GPT or Claude, prompted to return a label that it writes out as text for your code to parse. Laya is a dedicated decision model that scores each option you supply in one pass and returns probabilities. LLMs are more flexible; Laya is faster and cannot go off-list.

Most production "classification" in 2026 is a prompt. You describe the labels, paste the input, ask for JSON and parse the result. It works because frontier LLMs generalise well, and it is quick to build: no training data, no model hosting. The costs arrive later, as a per-token bill, a latency tail measured in seconds, and a parser that occasionally meets output it did not expect.

Laya takes the opposite route. It is an encoder-only decision model (ModernBERT-large for English, mmBERT-base for 100+ languages) with a small decision head on top. You send a state (text, an email, a ticket or JSON) and a set of typed questions, and it returns typed answers in one forward pass. It never generates text. Laya Studio is an independent hosted API powered by the open-source Laya model (Apache-2.0, published by Convai Innovations). It is not affiliated with Convai Innovations or with TypeSafe.

The two approaches are not interchangeable. An LLM is a general reasoner that happens to be able to classify. Laya is a narrow instrument built only for the classification step. The rest of this page covers where each one fits. For the model family itself, see What is Laya; for the architectural background, see encoder vs decoder models.

When you ask GPT or Claude for a label, the model generates tokens one at a time. The label is whatever string comes out, and structured-output modes constrain that string to a schema. Probabilities, if you want them, come from token logprobs. Multi-token labels make those awkward to turn into a clean distribution over your options, and some APIs do not expose logprobs at all.

Laya builds one input sequence per question:

[CLS] <type> question: instructions [SEP] [MASK] option 0 [MASK] option 1 ... [SEP] state [SEP]

Each option gets its own [MASK] marker. After the encoder and a two-layer decision head, a scorer reads the hidden state at every marker and produces one logit per option. The logits go through a softmax with a temperature fitted per question type and option count. The output is always a distribution over exactly the options you sent. The model cannot answer with a label that does not exist, misspell a key or return malformed JSON, because it never writes text. This is the property the model card sums up as "nothing to parse and nothing to hallucinate". See option-marker scoring and hallucination-free decisions for more.

Three question types cover most decisions (see choice, score and noul):

• choice: pick one of N named options. You get the choice, per-option probabilities and a confidence.
• score: place the state on an ordered rubric. You get the expected level, the per-level probabilities and a legend.
• noul: a yes/no statement. You get P(true).

Every question in a request is answered in the same forward pass, so asking five things about a ticket costs one call, not five.

On latency the gap is large, and it is structural. A decoder has to process the prompt and then generate at least a few tokens, and a reasoning model generates many more. An encoder reads the input once and scores.

The Laya model card reports these figures on a Tesla T4, measured in-process (they exclude network time):

For LLMs, the independent decision-model benchmark (DMB) by nibzard measured end-to-end p50 latency for one request:

These numbers are not directly comparable. The LLM figures include the network round trip to a hosted API; the Laya figures are model time on a GPU. Calling Laya Studio over the internet adds your own round trip to that. Even so, the gap is between tens of milliseconds and hundreds of milliseconds to seconds, and it widens as you add questions: an LLM's output grows with every extra field, while Laya scores 50 questions in 337 ms on the multilingual checkpoint.

If a decision sits inside a user-facing request, or runs in a loop inside an agent, the difference decides whether you can afford to classify at every step at all. See latency budgets for agents.

LLM classification is billed per token. You pay for the instructions, the label descriptions and the input on every call, plus the output tokens, which are often priced several times higher than input. DMB measured the cost of 1,000 decisions on its 77-way banking suite:

Laya Studio bills input tokens only: 1 credit = 1 input token, and there are no output tokens because Laya generates no text. Every new workspace gets 5 free runs. Current prices are on the pricing page. Because Laya's weights are Apache-2.0, you can also run them yourself and pay only for hardware. For a fuller treatment of the arithmetic, see LLM-as-classifier cost.

Cost also comes from engineering time outside the model bill: retry logic for malformed JSON, prompt tuning when a model version changes, and the evaluation harness you need either way. Laya removes the parsing and retry layer entirely. It does not remove the need to evaluate on your own data.

Calibration means that answers given 80% probability are right about 80% of the time (see calibrated probabilities and expected calibration error). It is what lets you write if confidence > 0.9: act, else escalate and trust the threshold.

Here the honest picture favours the LLMs on one axis. DMB built forced-uncertainty items, questions whose inputs do not contain enough information to answer. Every LLM it tested admitted ignorance on 97.3–100% of them, with ECE between 0.039 and 0.122. Laya was not part of that benchmark, so there is no like-for-like Laya number.

What the Laya model card does say is that the checkpoints ship over-confident. Refitting one temperature per (question type, option count) on held-out data moves mean ECE from 0.466 to 0.081 on the English checkpoint and from 0.314 to 0.106 on the multilingual one. The multilingual checkpoint ships with no fitted temperatures at all. In practice:

• Fit temperatures on a few hundred labelled examples from your own traffic before you set thresholds. See temperature scaling.
• Gate on confidence, not on action.act_probability. The model card reports that the act head reads 1.0 for almost every input (AUROC 0.30, issue #185), while confidence reached AUROC 0.77 on the same items.
• Route non-English input to the multilingual checkpoint before inference. The English checkpoint scored 0.000 accuracy at 0.952 confidence on Khmer, so confidence alone cannot catch that failure. Laya Studio routes by script automatically; see language routing.

Laya's training objective, RLCD, rewards the model with strictly proper scoring rules, so honest probabilities maximise expected reward (see proper scoring rules and RLCD). That makes calibration the intended behaviour, but you still verify it on your own data.

There is no single winner on accuracy; it depends on the task shape.

Laya is competitive or better on small label sets where the decision is mostly about reading the input. On AG News (4 labels) the model card reports 0.950 routed. On DAIR Emotion (6 labels) it reports 0.595. On email spam and phishing, its BENCHMARKS.md reports 0.993 and 0.980 on the English checkpoint, though both sources were in the training mix.

LLMs win clearly on large label sets. DMB's 77-way banking suite put the LLMs between 70.9% (gpt-5.4-nano) and 81.3% (gpt-oss-120b). Laya scores 0.425 on Banking77 because a choice question's options share a fixed token budget: at 77 options each label gets only 3–4 tokens and stops being distinguishable. The model card's advice is to keep choice questions under about 20 options, raise head_max_len, or split the label set into a coarse-to-fine hierarchy. DMB also found that every LLM handled 512 options.

LLMs also win on anything that needs reasoning: arithmetic, comparing dates, following multi-hop instructions, applying a long policy document. Laya does no chain of thought. Its ordinal score type is the weakest primitive (SST-5 0.372), and its held-out toxicity moderation is 0.530, close to chance on a balanced split.

Zero-shot on unfamiliar workflows, the base Laya checkpoints are weak. On the typed-decisions benchmark they score 0.362 (English) and 0.342 (multilingual), below the 0.461 majority-class baseline. A checkpoint fine-tuned on that benchmark's training split reaches 0.766. A frontier LLM will usually do better than an un-tuned Laya on a new, complex rubric. Laya does better once it has been specialised.

The migration is mostly mechanical: turn the labels in your prompt into a choice question, and turn each yes/no field into a noul.

A typical LLM prompt:

Classify the ticket into one of: refund, technical_help, billing_question, cancellation, other.
Also say whether it is urgent (true/false). Reply as JSON {"intent": ..., "urgent": ...}.

The Laya Studio equivalent:

curl -s https://api.laya.studio/v1/systemone \
  -H "Authorization: Bearer lsk_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "state": {"message": "I was charged twice for March, please refund the duplicate today."},
    "questions": {
      "intent": {
        "type": "choice",
        "instructions": "What does the customer want in `message`?",
        "criteria": {
          "refund": "money returned or a duplicate charge reversed",
          "technical_help": "a bug, outage or integration problem",
          "billing_question": "a question about an invoice, plan or payment method",
          "cancellation": "wants to cancel or downgrade",
          "other": "none of the other options fits"
        }
      },
      "urgent": {"type": "noul", "instructions": "Does `message` communicate time pressure or a deadline?"}
    }
  }'

The response has the shape the Laya runtime returns (values below are illustrative):

{
  "model": "laya-rl-agent",
  "answers": {
    "intent": {
      "type": "choice",
      "choice": "refund",
      "probabilities": {"refund": 0.86, "technical_help": 0.02, "billing_question": 0.09, "cancellation": 0.02, "other": 0.01},
      "confidence": 0.61,
      "action": {"act_probability": 1.0}
    },
    "urgent": {"type": "noul", "noul": 0.83, "confidence": 0.83, "action": {"act_probability": 1.0}}
  },
  "usage": {"input_tokens": 231, "output_tokens": 0},
  "routing": {"model": "english", "repo": "convaiinnovations/laya", "reason": "English Latin text"}
}

confidence is the normalised entropy of the distribution, 1 - H(p)/log(k). That is why it can read lower than the top probability. A TypeScript caller that keeps the LLM as a fallback:

const res = await fetch('https://api.laya.studio/v1/systemone', {
  method: 'POST',
  headers: { Authorization: `Bearer ${process.env.LAYA_API_KEY}`, 'Content-Type': 'application/json' },
  body: JSON.stringify({ state, questions }),
});
const { answers } = await res.json();
const intent = answers.intent;
if (intent.confidence >= threshold) {
  route(intent.choice);          // fast path: most traffic
} else {
  route(await askLlm(state));    // slow path: the uncertain tail
}

This cascade is the pattern we recommend. Laya handles the bulk at encoder latency and escalates low-confidence items to the LLM. The threshold should come from your own calibration data. See act or escalate routing and the agent tool routing and support ticket triage use cases.

Classification often runs on sensitive text: patient messages, HR cases, insurance claims, customer emails. With a hosted LLM API, where that text is processed and how long it is kept depend on the provider and your plan, so check its data terms.

With Laya Studio (details on our Swiss data residency page):

• Requests are answered on dedicated GPUs located in Switzerland, and every API response says where it was processed in the x-laya-region header.
• Zero content retention. The text and questions you send are processed in memory and discarded when the answer is returned. They are never written to a database or log, and never used to train anything.
• Swiss-only mode (one switch per workspace, or the x-laya-residency: ch header per request) means requests are only ever answered in Switzerland. If the Swiss pool is unavailable you get an error, never a silent detour abroad.
• Account data (accounts, API keys stored only as SHA-256 hashes, credit balances, usage metadata) lives in a Postgres database in the AWS Zurich region (eu-central-2). Request metadata only (time, status, number of questions, latency) is kept for 30 days.

Self-hosting the open Laya weights keeps data entirely on your own infrastructure. Laya Studio is designed to support compliance with the Swiss nFADP and the EU GDPR, and a Data Processing Agreement is available on request. It holds no formal certification (such as ISO 27001) today and does not sign HIPAA BAAs. You remain responsible for your legal basis to process personal and health data.

Choose Laya when:

• The decision has a small, stable label set (under ~20 options per question) and runs at volume.
• Latency matters: inline in a request, in an agent loop, or on a stream.
• You ask several questions about the same input and want them in one call.
• You need structured output that cannot fail to parse.
• You want open weights, so you can self-host or fine-tune for a specific workflow.

Keep the LLM when:

• The label set is large (dozens to hundreds of options) and you cannot restructure it into a hierarchy.
• The decision needs reasoning, arithmetic or a long policy applied step by step.
• You need a written rationale for each decision.
• Volume is low enough that per-token cost and seconds of latency do not matter.
• Honest "I don't know" behaviour on underspecified inputs is critical and you have not yet calibrated Laya on your data.

Use both in most real systems: Laya as the first-pass router and gate, and the LLM for the tail. To try it, create a free account and send your existing label set; the docs cover the request format in full.