Laya vs a fine-tuned BERT classifier: which should you use?

A fine-tuned BERT classifier is an encoder trained on labelled examples, with one fixed output per label. Laya uses the same family of encoder but reads its labels from each request, answers several typed questions per input in one pass, and is trained with proper scoring rules. BERT suits fixed tasks; Laya suits changing ones.

If you have shipped a text classifier in the last few years, it was probably a BERT-family encoder with a linear layer on top, trained with cross-entropy on a few thousand labelled examples. That recipe is well understood (Devlin et al., 2019). It is cheap to serve and, on a stable task with enough data, hard to beat.

Laya is also an encoder. The English checkpoint is ModernBERT-large, fully fine-tuned, with a decision head trained from scratch, 421M parameters in total. The multilingual checkpoint is mmBERT-base at 322M. So the comparison is not "old BERT vs a new architecture". It is about three design choices on top of the same kind of backbone:

1. Where the labels live. In a BERT classifier they are baked into the output layer. In Laya they arrive in the request.
2. How many tasks one model serves. A BERT head does one task. Laya answers any mix of choice, score and noul questions about one input in one pass.
3. What the training objective rewards. Laya is trained with RLCD, reinforcement learning against strictly proper scoring rules. A BERT classifier uses cross-entropy.

Laya Studio is an independent hosted API powered by the open-source Laya model. It is not affiliated with its authors, Convai Innovations. Background reading: encoder vs decoder models, ModernBERT and mmBERT.

A standard fine-tuned classifier reads the [CLS] vector and multiplies it by a weight matrix with one row per label. Label 17 is row 17. The model has no idea what label 17 means except through the examples it was trained on, which is why adding a label or renaming one means retraining.

Laya puts the options into the input:

[CLS] choice question: Which team should handle this? [SEP]
  [MASK] billing: invoices, payments, refunds
  [MASK] technical: bugs, outages, system errors
  [MASK] sales: pricing, new contracts [SEP]
  <state> [SEP]

The encoder attends across the instructions, every option description and the state at once. A two-layer transformer head then refines the representation, and a scorer reads one logit at each [MASK] position. The softmax over those logits is the answer distribution. Because the options are text, the model can use their descriptions, and you can change the label set in the next request. The model card puts it this way: "The answer space is defined at request time, so new schemas need no retraining." See option-marker scoring.

The trade-off is a token budget. All options share head_max_len (192 tokens on the English checkpoint, 256 on multilingual). With 77 options each label gets about 3–4 tokens, and Laya's Banking77 accuracy falls to 0.425. A linear head has no such limit: it can have 77 or 700 outputs at no extra input cost. If your task is one large, fixed taxonomy, that is a real advantage for the classic approach.

The classic recipe needs labelled data before it produces anything useful: typically hundreds of examples per class for a reliable model, plus a validation set. Few-shot methods such as SetFit lower that requirement considerably, but you still train a model per task.

Laya gives you an answer on the first request, with no training. That makes it useful for prototyping a schema, for tasks whose labels change weekly, and for the long tail of small decisions that never justified a dedicated model.

That said, the zero-shot results need stating plainly. On the typed-decisions benchmark (400 cases, 2,000 decisions across four business workflows), the base checkpoints are near chance:

A task-specific fine-tuned encoder beats base Laya by a wide margin on that benchmark. The model card's own summary is: "Laya is a fast base to specialise, not a zero-shot decision engine." Zero-shot Laya does well on tasks close to its training mix, such as spam, phishing, topic and intent, and less well on novel multi-field business rubrics. See zero-shot vs fine-tuned.

The most useful comparison is fine-tuned Laya against a fine-tuned BERT on the same data. The model card has one published data point. On typed-decisions, the published ModernBERT-base specialist scores 0.646. laya-typed-decisions, fine-tuned from the English Laya checkpoint on the benchmark's 1,200-case training split (6,000 decisions), scores 0.766, above the 0.735 teacher self-agreement ceiling. It wins all four workflows: invoice processing 0.804, security incidents 0.766, customer service 0.764 and agent-trace observability 0.730.

Some of that gap is backbone size (large vs base). Some is the head: one fine-tuned Laya model answers all 2,000 decisions, spread across noul, choice and score questions with different label sets. With the classic approach you would train several heads or several models. The fine-tuning run took about 4–5 hours on Kaggle's free 2xT4, and the notebook is public in the Laya repository.

Two caveats from the same model card:

• The fine-tuned checkpoint is a specialist: "Expect it to behave like the base laya checkpoint, or worse, on anything else."
• Its per-type temperatures were fitted on training data (issue #186), so treat its confidence as uncalibrated until you refit.

Laya Studio serves the three published checkpoints. It selects typed-decisions only when you ask for it with "model": "typed-decisions". Custom fine-tunes are a self-hosting path today, using the open weights.

Cross-entropy is the log score, which is already a strictly proper scoring rule. A classifier trained to convergence with it is, in principle, rewarded for honest probabilities. In practice, modern deep networks tend to come out over-confident (Guo et al., 2017), and the standard fix is post-hoc temperature scaling on a held-out set.

Laya's RLCD objective combines the log score with a spherical score, and adds a ranked probability score for ordinal score questions. Exploration noise is added to the logits, and the update is REINFORCE with a group-mean baseline. See RLCD and proper scoring rules. The ranked probability score matters for ordinal rubrics: it penalises predicting "critical" when the truth is "low" more than predicting "medium".

Neither approach removes the need to calibrate on your own data. The Laya model card is explicit that the checkpoints ship over-confident. Refitting one temperature per (question type, option count) moves mean ECE from 0.466 to 0.081 on the English checkpoint and from 0.314 to 0.106 on the multilingual one. The runtime clamps temperatures to [0.5, 5.0] after a shipped bucket was found to sharpen probabilities about tenfold. The workflow is the same as for a BERT classifier: hold out labelled data, fit temperatures, then set thresholds. See temperature scaling and expected calibration error.

A base-size BERT classifier (around 110M parameters) is smaller and usually faster than Laya's 421M English checkpoint. If you need the absolute lowest per-item latency on CPU for a single fixed task, a small dedicated model is hard to beat.

Laya's latency figures, from the model card (Tesla T4, in-process):

On CPU, the preloaded router runs at 193–464 ms per request.

The operational comparison changes once you have more than one task. Ten classifiers means ten training pipelines, ten model versions to monitor and ten deployments, or one multi-head model you had to design yourself. With Laya those ten tasks become ten questions in one request, answered by one model in one pass. On Laya Studio you do not host anything: POST https://api.laya.studio/v1/systemone, billed per input token (1 credit = 1 input token), with 5 free runs. Prices are on the pricing page.

import requests

resp = requests.post(
    "https://api.laya.studio/v1/systemone",
    headers={"Authorization": "Bearer lsk_live_..."},
    json={
        "state": {"subject": "Duplicate charge on invoice #4411", "body": "We were billed twice for March."},
        "questions": {
            "department": {"type": "choice", "instructions": "Which department should handle this?",
                           "criteria": {"billing": "invoices, payments, refunds", "technical": "bugs, outages",
                                        "sales": "pricing, new contracts", "other": "everything else"}},
            "urgency": {"type": "score", "instructions": "How urgent is this request?",
                        "criteria": ["not urgent", "soon", "critical deadline or blocking issue"]},
            "refund_requested": {"type": "noul", "instructions": "Does the sender ask for money back?"},
        },
    },
    timeout=10,
)
answers = resp.json()["answers"]
print(answers["department"]["choice"], answers["urgency"]["score"], answers["refund_requested"]["noul"])

A score answer returns the probability-weighted level (for example 1.84 on a 0–2 scale), with the per-level probabilities and a legend. score is Laya's weakest primitive (SST-5 0.372), so validate it before you rely on it. A noul returns P(true). If a noul looks stuck on your data (issue #156), ask the same thing as a two-option choice with neutral keys.

Train a dedicated BERT classifier when:

• You have one task with a large, stable taxonomy (dozens to hundreds of labels).
• You have, or can buy, thousands of labelled examples.
• You need the smallest possible model for CPU or edge inference.
• The label set changes rarely enough that retraining is not a burden.

Use Laya when:

• The label set is small and changes often, or you are still discovering it.
• You ask several questions about each input: intent, urgency, sentiment, risk flags.
• You want answers before you have training data, then a path to specialise.
• You want one model and one API instead of a fleet of task heads.

Fine-tune Laya when you have a defined multi-question workflow and labelled data. That is where the published results are strongest (0.766 against 0.646 for the published ModernBERT-base specialist on typed-decisions). Keep the limits in mind: under ~20 options per choice question, calibrate before thresholding, and gate on confidence rather than action.act_probability, which carries no usable signal yet.

To start without training anything, sign up and send your schema. The docs describe every question type. For the underlying ideas, see typed decisions and decision models for AI agents.