Autoregressive model

An autoregressive model factorises the probability of a sequence as a product of next-token probabilities, p(x) = Π_t p(x_t | x_<t), and generates by sampling one token, appending it, and repeating. Every modern chat LLM is autoregressive, built on a decoder-only transformer with causal attention.

This design is what makes LLMs so flexible: they can produce any text, explain their reasoning, follow novel instructions and call tools. The same design carries costs when all you need is a label:

• Latency scales with output length. Each output token is another forward step. A JSON answer with a few fields can take dozens of steps, plus time-to-first-token.
• Output must be parsed. The model can return a label you did not offer, invalid JSON, or an explanation instead of an answer.
• Probabilities are awkward. The "confidence" of a label is the product of several token probabilities, affected by tokenisation, and often unavailable through hosted APIs. Verbalised confidence ("I am 90% sure") is not a calibrated probability.
• Cost is per token, input and output.

Autoregressive vs Laya

Laya is the opposite design: a non-autoregressive encoder that scores every option you supply in one pass and returns a softmax distribution over them. It cannot write, summarise or reason in multiple steps. It is intended for the high-volume, fixed-answer-space decisions in front of or around an LLM: triage, routing, guardrails, moderation. A common pattern is to let Laya make the fast System 1 call and escalate uncertain cases to an autoregressive System 2 model.

See Laya vs LLM classifiers and the cost of using an LLM as a classifier.