What is Laya? The open-source AI model for fast, calibrated decisions

Laya is an open-source AI decision model. You give it a piece of text or data plus a few typed questions (pick one option, rate on a scale, or answer yes or no), and it returns an answer to each with a probability, in one pass. It never generates text, so every answer comes from your own list of options.

Laya is a decision model, not a chat model. You give it a state, meaning anything your software has on hand: a support ticket, an inbound email, a chat transcript or a JSON record. You also give it a set of typed questions, such as "which team should handle this?", "how urgent is it?" and "is the customer asking for a refund?". It returns one typed answer per question with a probability for every possible answer. It never writes text. The model card puts it plainly: "It never generates text, so there is nothing to parse and nothing to hallucinate."

The model is published by Convai Innovations on Hugging Face at convaiinnovations/laya under the Apache-2.0 license. The Python package is laya on PyPI (version 0.3.7 at the time of writing), and the source lives at github.com/NandhaKishorM/laya. The license permits commercial use, self-hosting, fine-tuning and redistribution.

The design mirrors a category TypeSafe AI calls System One models: fast, typed, probabilistic judgments that code can branch on. TypeSafe's hosted model in that category is Jev (see /learn/what-is-jev). Laya's package ships an HTTP server that speaks the same /v1/systemone request and response shape as Jev, so a client written for one can point at the other by changing its base URL. The trade-offs are covered in /compare/laya-vs-jev.

Laya Studio, the site you are reading, is an independent hosted API powered by the open-source Laya model. It is not affiliated with or endorsed by Convai Innovations or TypeSafe. We run the checkpoints, route requests between them, and bill per input token, 30% below Jev's list price. Everything this page says about the model itself comes from the public model cards and package source, and the sources are listed at the bottom.

An LLM answers a classification question by generating tokens one at a time, and your code parses the result. Laya works differently. It is a bidirectional encoder (see /learn/encoder-vs-decoder-models) with a small decision head on top, and it scores every candidate answer in parallel.

For each question, the package builds one token sequence in this format (from common.py):

[CLS] <type> question: <instructions> [SEP] [MASK] option0 [MASK] option1 ... [SEP] <state> [SEP]

Each option gets its own [MASK] token, called an option marker. The model reads the whole sequence bidirectionally, reads the hidden state at each marker, scores it with a small MLP and applies a softmax over that question's options. That gives one probability per option, and the highest one is the answer. Because the options are part of the input rather than a fixed output layer, the answer space is defined at request time. A new label set needs no retraining, only a new request. /learn/option-marker-scoring has the details.

Every question in a call becomes one row of a batch, and all rows go through the network together. The model card puts it as "every question in a call is answered in one single forward pass." That is why five questions cost far less than five times one question. On a T4, one question takes 39.5 ms on the English checkpoint, and ten questions take 158.6 ms, or 15.9 ms per question.

The architecture, per the model card:

Two consequences follow from this design. First, there is no decoding loop, so latency is roughly one encoder pass no matter how many options a question has. Second, the question and its options share a fixed token budget (head_max_len). That explains Laya's best-known weakness, very large label sets, which the limitations section below covers.

The model also computes a separate act/escalate output, action.act_probability. It was designed to tell your code whether to act or escalate. The model card says plainly that this output "carries no usable signal yet" (issue #185), so gate on confidence instead. See /learn/act-escalate-routing.

Laya accepts exactly three question types (QTYPES = {"choice": 0, "score": 1, "noul": 2} in common.py). They match the three primitives of the Jev API, so the same question definitions work against either service. /learn/choice-score-noul goes deeper, and /learn/typed-decisions explains why typed outputs matter.

A few details from agent.py matter in practice:

• score returns an expected value, not an argmax. With three levels, score: 1.74 means the probability mass sits mostly on level 2 with some on level 1. The full probabilities map is there if you would rather take the argmax.
• confidence is normalised entropy: 1 - H(p) / log(k) over the k options. It is 1.0 when all the mass sits on one option and 0.0 when the distribution is uniform. For noul it is max(p, 1 - p).
• Probabilities are temperature-scaled per bucket. Before the softmax, logits are divided by a fitted temperature chosen by question type and option count (buckets 2, 3-5, 6-10, 11+). The package clamps temperatures to [0.5, 5.0] because one shipped bucket (choice:11+, 0.1006) sharpened a 0.24 top probability to 0.99. More on this in /learn/temperature-scaling.
• Malformed questions fail loudly. A choice question without criteria, or an unknown type, raises an error naming the question. the self-hosted server returns it as an HTTP 422.

Here is a real request body, adapted from the model card quickstart:

{
  "state": {
    "from": "user@acme.com",
    "subject": "Duplicate charge on invoice #4411",
    "body": "Hi, we were billed twice for March. Please refund the duplicate today or we will cancel our plan."
  },
  "questions": {
    "department": {
      "type": "choice",
      "instructions": "Which department should handle this request?",
      "criteria": {
        "billing": "invoices, payments, refunds",
        "technical": "bugs, outages, system errors",
        "sales": "pricing, new contracts",
        "other": "everything else"
      }
    },
    "urgency": {
      "type": "score",
      "instructions": "How urgent is this request?",
      "criteria": ["not urgent", "soon", "critical deadline or blocking issue"]
    },
    "churn_risk": {
      "type": "noul",
      "instructions": "Does the user threaten to cancel or leave?"
    }
  }
}

The Laya family has three checkpoints. All of them live in the convaiinnovations/laya hub repo (the English one at the root, the other two in subfolders) and in their own standalone repos:

Laya's benchmarks give a strong reason to route. The English checkpoint does not degrade gracefully outside English. It collapses, and it stays confident while it does. Across all 51 MASSIVE languages on 20-option intent classification (random = 0.050), the multilingual model card reports these results:

On Khmer, the English checkpoint scores 0.000 accuracy at 0.952 confidence, and its mean confidence never drops below 0.885. A confidence threshold cannot catch this failure, so the language decision has to happen before the forward pass.

The package's Router does exactly that. It inspects the Unicode script of the state in under 0.5 ms of pure Python (lang.py covers 25 script families). It sends non-Latin scripts to the multilingual checkpoint. For Latin-script text it uses a stopword and diacritic heuristic to decide whether the text is English. Version 0.3.7 improved routing for Spanish, Italian, Portuguese and French whose accents were stripped. On MASSIVE with accents stripped, Italian utterances of six or more words routed correctly went from 39% to 80%.

Routing takes the better column in every row. The typed-decisions checkpoint is never chosen automatically: it is a specialist fine-tuned on four synthetic workflows, and the router's docstring says it "should not be a silent default."

Laya Studio runs this router on every request and returns its decision in a routing field, so you can log which checkpoint answered and why. To pin a checkpoint, pass "model": "english", "multilingual" or "typed-decisions". See /learn/language-routing and /learn/multilingual-classification.

Most classifiers are trained with cross-entropy against a one-hot label. Laya's checkpoints are trained with RLCD, Reinforcement Learning for Calibrated Decisions. TypeSafe also uses this name for Jev's training. Laya's model card describes its own recipe:

1. The policy reports a probability distribution over the options.
2. Exploration adds zero-mean Gaussian noise to the logits.
3. The reward is a strictly proper scoring rule: log score plus spherical score, plus the ranked probability score (RPS) for ordinal score questions.
4. Updates use REINFORCE with a group-mean baseline (GRPO-style).
5. Multi-turn conversations use TD(λ = 1.0) over prefix slices.

The reward function is public in common.py as proper_reward: log score, plus 0.5 × spherical score, minus RPS on score questions. A scoring rule is strictly proper when the only way to maximise expected reward is to report your true belief. Overstating confidence loses reward on the cases you get wrong, and understating it loses reward on the cases you get right. /learn/proper-scoring-rules and /learn/rlcd-reinforcement-learning-calibrated-decisions cover the theory.

Training on a proper scoring rule does not mean the shipped probabilities are calibrated out of the box. The model card says the checkpoints "ship over-confident." Refitting one temperature per (question type, option count) on held-out data moves mean expected calibration error from 0.466 to 0.081 on the English checkpoint and from 0.314 to 0.106 on the multilingual one. The multilingual checkpoint ships with no fitted temperatures at all. Before you set confidence thresholds in production, fit temperatures on a few hundred of your own labelled examples. See /learn/calibrated-probabilities and /learn/expected-calibration-error.

The multilingual checkpoint was trained from scratch with RLCD: 15,987 updates over 4 epochs, about 4.97 hours. The typed-decisions checkpoint was fine-tuned from the English one on the benchmark's 1,200-case training split (6,000 decisions), using RLCD plus soft cross-entropy against the teacher's distributions. The reproduction notebook takes about 4 to 5 hours on Kaggle's free 2xT4.

All numbers below come from the Laya model cards and BENCHMARKS.md. They were measured on a Tesla T4, in-process (no network), with every checkpoint answering byte-identical questions.

Latency

That works out to 103 to 332 questions per second batched on one T4. On CPU, the preloaded router answers in 193 to 464 ms. These are model-side numbers: a hosted API such as Laya Studio adds network round-trip time on top. /learn/latency-budgets-for-agents covers how to budget for that.

Accuracy on public datasets

The Laya card notes that its Jev figures are "third-party published, never measured here," and that sample sizes and prompts differ, so treat the comparison as indicative.

Application themes (400 cases each)

Read this table carefully. Spam and phishing are strong, but both were in the training mix. Held-out toxicity moderation at 0.530 is barely above chance on a balanced split, and the benchmark file says so: "hand-picked examples work, real traffic does not." Before you trust any of these tasks in production, run your own evaluation on your own data.

The Laya model cards include a detailed "Honest Limits" section. These are the limits that matter most when you decide whether to use it:

• Large label sets. All options in a question share one token budget (head_max_len: 192 on English, 256 on multilingual). With 77 options, each label gets about 3 to 4 tokens, and the labels stop being distinguishable. Both base checkpoints score exactly 0.425 on Banking77, against Jev's published 0.870 on 72 labels. Keep choice questions under about 20 options, or split them into a coarse-to-fine hierarchy. The package includes an opt-in embedding shortlist (predict_shortlist, default k = 20) for this.
• Zero-shot on complex workflows. The base checkpoints score 0.362 (English) and 0.342 (multilingual) on typed-decisions. That is below the 0.461 majority-class baseline. The 0.766 figure belongs to the checkpoint fine-tuned on that benchmark's own training split. In the card's words: "Laya is a fast base to specialise, not a zero-shot decision engine."
• score is the weakest primitive. SST-5 ordinal sentiment is 0.372 on the English checkpoint and 0.282 on the multilingual one. The multilingual checkpoint also has a measured position bias on score questions: it rarely picks the first-listed level (issue #131).
• noul can follow its labels instead of the state (issue #156). Sometimes it returns a confident "no" on clearly positive input. The workaround is to ask a two-option choice with neutral keys (A/B) and put your yes/no wording in the descriptions.
• action.act_probability carries no usable signal yet (issue #185). It reads 1.0 on almost every input, and its raw logits run against correctness (AUROC 0.30). Gate on confidence instead, which reaches an AUROC of 0.77 on the same items.
• Over-confidence as shipped. Refit temperatures on your own data before you trust the probabilities as calibrated.
• Low-resource languages. Routing fixes the collapse, but it does not make every language good. On the multilingual checkpoint, Swahili scores 0.210, Tamil 0.250 and Amharic 0.110 on 20-option MASSIVE intent.
• Short context. The English checkpoint leaves about 320 tokens for state. Long emails and documents are truncated, so trim quoted replies and boilerplate first (the package ships clean_email_body for this).

None of these is hidden. If your task needs 70+ labels in one question, or strong zero-shot accuracy on a complex bespoke rubric, test a larger model or Jev before committing. /learn/zero-shot-vs-fine-tuned covers the trade-off.

There are two ways to run Laya.

Self-host. pip install "laya[serve]", then run laya-serve. It serves POST /v1/systemone and GET /health on port 8000. With no key set it binds 0.0.0.0 with no authentication, so set LAYA_API_KEY before exposing it. You manage the GPU, preloading, thread counts (LAYA_THREADS matters a lot on CPU) and temperature refits.

Laya Studio. We host the same open checkpoints behind a managed endpoint with API keys, automatic language routing and usage metering. The request is Jev-wire-compatible:

curl -s https://api.laya.studio/v1/systemone \
  -H "Authorization: Bearer lsk_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "state": {"body": "We were billed twice for March. Refund the duplicate today or we cancel."},
    "questions": {
      "department": {"type": "choice", "instructions": "Which department should handle this?",
        "criteria": {"billing": "invoices, payments, refunds", "technical": "bugs, outages",
                     "sales": "pricing, new contracts", "other": "everything else"}},
      "urgency": {"type": "score", "instructions": "How urgent is this request?",
        "criteria": ["not urgent", "soon", "critical deadline or blocking issue"]},
      "churn_risk": {"type": "noul", "instructions": "Does the user threaten to cancel or leave?"}
    }
  }'

The response has the shape produced by agent.py plus the router's decision. The values below are illustrative:

{
  "model": "laya-rl-agent",
  "answers": {
    "department": {
      "type": "choice",
      "choice": "billing",
      "probabilities": {"billing": 0.94, "technical": 0.03, "sales": 0.02, "other": 0.01},
      "confidence": 0.7924,
      "action": {"act_probability": 1.0}
    },
    "urgency": {
      "type": "score",
      "score": 1.74,
      "legend": {"0": "not urgent", "1": "soon", "2": "critical deadline or blocking issue"},
      "probabilities": {"0": 0.04, "1": 0.18, "2": 0.78},
      "confidence": 0.4254,
      "action": {"act_probability": 1.0}
    },
    "churn_risk": {
      "type": "noul",
      "noul": 0.89,
      "confidence": 0.89,
      "action": {"act_probability": 1.0}
    }
  },
  "usage": {"input_tokens": 431, "output_tokens": 0},
  "routing": {
    "model": "english",
    "repo": "convaiinnovations/laya",
    "reason": "English Latin text",
    "workflow": null
  }
}

The routing object also carries a detection block with the script and language analysis. output_tokens is always 0, because nothing is generated.

In Python, using plain requests:

import os, requests

resp = requests.post(
    "https://api.laya.studio/v1/systemone",
    headers={"Authorization": "Bearer " + os.environ["LAYA_API_KEY"]},
    json={
        "state": "Mein Konto wurde zweimal belastet. Bitte erstatten Sie den Betrag.",
        "questions": {
            "refund_requested": {"type": "noul", "instructions": "Does the sender ask for money back?"}
        },
    },
    timeout=10,
)
data = resp.json()
print(data["routing"]["model"])                       # multilingual
print(data["answers"]["refund_requested"]["noul"])    # probability the answer is yes

In TypeScript, using fetch:

const res = await fetch('https://api.laya.studio/v1/systemone', {
  method: 'POST',
  headers: {
    Authorization: 'Bearer ' + process.env.LAYA_API_KEY,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    state: { subject: 'Invoice #4411', body: 'Charged twice, please refund.' },
    questions: {
      team: { type: 'choice', instructions: 'Which team handles this?', criteria: ['billing', 'technical', 'sales'] },
    },
  }),
});
const { answers, routing } = await res.json();
if (answers.team.confidence > 0.6) routeTo(answers.team.choice);
else sendToHuman();

Billing. One credit equals one input token. Each question reads the state once, so the three-question request above costs about three times the state's tokens plus the questions' wording. Every new workspace gets 5 free runs. Current plans are on /pricing, the full reference is in /docs, and you can create a key at /signup.

If you self-host Laya, your data never leaves your own infrastructure. If you call Laya Studio, this is what happens to it, as set out on our Swiss data residency page:

• Processed in Switzerland. Our primary inference pool runs on dedicated GPUs located in Switzerland. Every API response says where it was processed in the x-laya-region header.
• Zero content retention. The text and questions you send are processed in memory and discarded when the answer is returned. They are never written to a database or log, and never used to train anything.
• Swiss-only mode. Turn on one switch per workspace, or send the x-laya-residency: ch header per request, and requests are only ever answered in Switzerland. If the Swiss pool is unavailable you get an error, never a silent detour abroad.
• Account data in Zurich. Accounts, API keys (stored only as SHA-256 hashes), credit balances and usage metadata live in a Postgres database in the AWS Zurich region (eu-central-2). Request metadata only (time, status, number of questions, latency) is kept for 30 days for billing and debugging.

No retention plus Swiss processing makes Laya Studio a fit for sensitive text such as patient messages, clinical intake notes, HR cases or financial correspondence. It is designed to support compliance with the Swiss Federal Act on Data Protection (nFADP) and the EU GDPR, and a Data Processing Agreement is available on request. You remain responsible for your legal basis to process personal and health data. Laya Studio holds no formal certification (such as ISO 27001) today and does not sign HIPAA BAAs.

Laya fits the high-volume, narrow judgments that sit inside ordinary code: the "smart if-statements" of a pipeline.

Good fits

• Ticket and email triage: department, urgency, refund requested, churn risk. See /use-cases/support-ticket-triage and /use-cases/email-routing.
• Intent detection and agent tool routing with a modest number of options. See /use-cases/intent-detection and /use-cases/agent-tool-routing.
• Pre-LLM guardrails such as jailbreak and prompt-injection flags, used as one signal among several.
• Multilingual intake, where automatic routing matters. See /use-cases/multilingual-intake.
• Workloads that must run on your own hardware under a permissive license, or where per-token API costs dominate.

Poor fits

• Anything that needs generated text: replies, summaries, extraction of free-form spans.
• Single questions with 50+ labels, unless you shortlist or build a hierarchy.
• Complex bespoke rubrics where you need strong zero-shot accuracy and cannot fine-tune.
• Numeric reasoning, date arithmetic and counting. Do these in code.

Compared with the alternatives: an LLM used as a classifier (see /compare/laya-vs-llm-classifiers and /learn/llm-as-classifier-cost) is more flexible but slower, more expensive per call, and its verbalised confidence is not trained against a proper scoring rule. A fine-tuned BERT (see /compare/laya-vs-fine-tuned-bert) is usually more accurate on one fixed label set, but it needs labelled data and a retrain whenever the labels change. Jev (/compare/laya-vs-jev) is a managed, closed model on the same wire protocol, with a larger context and a 255-option limit.

The practical approach is to send your real traffic to a free Laya Studio key, compare the answers with your labels, fit thresholds on confidence, and decide from your own numbers.