Non-autoregressive model

A non-autoregressive model computes its entire output in a single pass (or a fixed number of passes) rather than emitting one token at a time with each step conditioned on the last. The term comes from machine translation, where non-autoregressive decoders were proposed to escape the sequential bottleneck of autoregressive generation. In classification the idea is older and simpler: an encoder reads the input once and a head produces scores for every label at the same time.

The trade-off is expressiveness for speed and structure. A non-autoregressive model cannot write free-form text, reason step by step, or invent an answer outside its output space. In exchange its latency does not grow with output length, its output is always well-formed, and every probability comes from one forward computation rather than a product of sampled tokens.

Laya as a non-autoregressive decision model

Laya never generates text. Each call packs every question you send into one batch and runs a single forward pass through the encoder and decision head. Every option gets a logit at its option marker, and the answer is read directly from the resulting distribution. The response reports output_tokens: 0, which is literally true.

Consequences you can rely on:

• Latency is flat in answer length. One question takes 39.5 ms on the English checkpoint and 32.8 ms on the multilingual one (T4 GPU, per the model card).
• The answer is always one of your options. There is nothing to parse and nothing to hallucinate in the sense of an unrequested label or malformed JSON.
• It cannot think longer on hard inputs. That is a System 2 job.

See non-autoregressive models and System 1 vs System 2 AI.