Context window

A model's context window is the maximum sequence length, in tokens, that it can attend over in one forward pass. For generative LLMs it covers the prompt plus the generated output; for an encoder it covers the whole input. Anything beyond the limit has to be truncated, summarised or split.

A long context is not free: attention cost grows with length, and models do not use all positions equally well. For classification, the relevant question is usually not "how long can it go?" but "does the evidence I need fit?".

Context in Laya

Each Laya question is its own sequence, laid out as:

[CLS] <type> question: instructions [SEP] [MASK] option0 [MASK] option1 … [SEP] state [SEP]

The budget splits into two parts:

Show technical detailsHide technical details· 3 rows × 4 columns

checkpoint	max_len	head_max_len (question + options)	left for state
English (ModernBERT-large)	512	192	about 320
multilingual (mmBERT-base)	1,024	256	about 768
typed-decisions	1,024	256	about 768

Two consequences follow. First, state text past the budget is cut from the end, so put the most decision-relevant fields first, clean quoted email history, and prefer structured JSON over raw dumps. Second, options compete for the head budget. With 77 options, each gets only 3–4 tokens, which is why Banking77 accuracy falls to 0.425. Each option is also capped at 48 tokens.

The mmBERT encoder supports up to 8k tokens and self-hosters can raise head_max_len and max_len at runtime, at a latency cost. See option-marker scoring and ModernBERT.