XNLI

XNLI (Cross-lingual Natural Language Inference) is a benchmark for natural language inference across languages. It takes the MultiNLI setup (premise, hypothesis, and a label of entailment, neutral or contradiction) and provides professionally translated development and test sets in 15 languages, including English, French, Spanish, German, Greek, Bulgarian, Russian, Turkish, Arabic, Vietnamese, Thai, Chinese, Hindi, Swahili and Urdu.

It is widely used to measure cross-lingual transfer: train or tune on English, then test how well the model reasons in other languages. Because every language has the same items, per-language scores are directly comparable.

Laya's XNLI results

The Laya model card reports XNLI accuracy for both general checkpoints on a shared benchmark run:

Two readings:

1. On English, the English checkpoint (ModernBERT-large) is slightly better, so there is no reason to route English text to the multilingual model.
2. Off English, the multilingual checkpoint (mmBERT-base) is about 21 points better. The English model's drop is steepest on non-Latin scripts.

This table is a main reason the Laya Router exists: taking the better checkpoint per language gives 0.860 and 0.731, better than either model alone. See language routing.

XNLI items are three-way choices, which Laya expresses as a choice question with the premise and hypothesis in the state. See multilingual classification and the multilingual intake use case.