Glossary
Context window
The context window is the maximum amount of text, counted in tokens, that a model can read in one go; anything longer is cut off or must be split.
Swiss-hosted inference. Nothing you send is ever stored.Swiss data residency
What Context window means
A model's context window is the maximum sequence length, in tokens, that it can attend over in one forward pass. For generative LLMs it covers the prompt plus the generated output; for an encoder it covers the whole input. Anything beyond the limit has to be truncated, summarised or split.
A long context is not free: attention cost grows with length, and models do not use all positions equally well. For classification, the relevant question is usually not "how long can it go?" but "does the evidence I need fit?".
Context in Laya
Each Laya question is its own sequence, laid out as:
[CLS] <type> question: instructions [SEP] [MASK] option0 [MASK] option1 … [SEP] state [SEP]
The budget splits into two parts:
Show technical detailsHide technical details· 3 rows × 4 columns
| checkpoint | max_len | head_max_len (question + options) | left for state |
|---|---|---|---|
| English (ModernBERT-large) | 512 | 192 | about 320 |
| multilingual (mmBERT-base) | 1,024 | 256 | about 768 |
| typed-decisions | 1,024 | 256 | about 768 |
Two consequences follow. First, state text past the budget is cut from the end, so put the most decision-relevant fields first, clean quoted email history, and prefer structured JSON over raw dumps. Second, options compete for the head budget. With 77 options, each gets only 3–4 tokens, which is why Banking77 accuracy falls to 0.425. Each option is also capped at 48 tokens.
The mmBERT encoder supports up to 8k tokens and self-hosters can raise head_max_len and max_len at runtime, at a latency cost. See option-marker scoring and ModernBERT.
Related terms
How Context window connects to the rest of the vocabulary.