Glossary
XNLI
XNLI is a public benchmark that tests whether a model can tell if one sentence follows from another across 15 languages, not just English.
Swiss-hosted inference. Nothing you send is ever stored.Swiss data residency
What XNLI means
XNLI (Cross-lingual Natural Language Inference) is a benchmark for natural language inference across languages. It takes the MultiNLI setup (premise, hypothesis, and a label of entailment, neutral or contradiction) and provides professionally translated development and test sets in 15 languages, including English, French, Spanish, German, Greek, Bulgarian, Russian, Turkish, Arabic, Vietnamese, Thai, Chinese, Hindi, Swahili and Urdu.
It is widely used to measure cross-lingual transfer: train or tune on English, then test how well the model reasons in other languages. Because every language has the same items, per-language scores are directly comparable.
Laya's XNLI results
The Laya model card reports XNLI accuracy for both general checkpoints on a shared benchmark run:
| English checkpoint | multilingual checkpoint | |
|---|---|---|
| XNLI, English | 0.860 | 0.843 |
| XNLI, 14 other languages | 0.521 | 0.731 |
Two readings:
- On English, the English checkpoint (ModernBERT-large) is slightly better, so there is no reason to route English text to the multilingual model.
- Off English, the multilingual checkpoint (mmBERT-base) is about 21 points better. The English model's drop is steepest on non-Latin scripts.
This table is a main reason the Laya Router exists: taking the better checkpoint per language gives 0.860 and 0.731, better than either model alone. See language routing.
XNLI items are three-way choices, which Laya expresses as a choice question with the premise and hypothesis in the state. See multilingual classification and the multilingual intake use case.
Related terms
How XNLI connects to the rest of the vocabulary.