Multilingual classification: one decision, many languages

Multilingual classification means assigning labels to text written in many languages with one system. It is harder than it looks because an English-trained classifier fails on other languages without warning: it may misread the text entirely yet still report high confidence. Reliable systems translate first, use multilingual or per-language models, or route between them.

A classifier that works on English does not degrade gracefully on other languages. It usually fails in one of three ways:

1. It cannot tokenise the text. An English tokenizer turns Devanagari, Hangul or Khmer into fragments the model never learned. The input is effectively noise.
2. It reads the words but misses the meaning. Latin-script languages share characters and some vocabulary with English, so an English model extracts partial signal from French or Spanish but misses idioms, negation and word order.
3. It stays confident while failing. This is the dangerous one. The model's probability outputs were calibrated on English. On unreadable input they do not drop toward uniform; they remain sharp.

The Laya model card quantifies all three for its English checkpoint on 20-option MASSIVE intent classification (random = 0.050): 0.783 on English, 0.487 on French, 0.480 on Spanish, 0.100 on Hindi, 0.103 on Korean, and 0.000 on Khmer at 0.952 reported confidence. The package source adds one more: on Brazilian Portuguese support text, the English checkpoint "reports 0.97 mean confidence at 0.47 accuracy." Confidence gating does not help when confidence itself is broken.

Cross-lingual transfer is what makes the multilingual encoder approach practical. Pires et al. (2019) showed that multilingual BERT fine-tuned on one language could classify others it never saw labelled examples in. XNLI (Conneau et al., 2018) became the standard benchmark for this: train on English natural language inference, test on 14 other languages. XLM-R and later mmBERT pushed the transfer gap down with more data and better training.

The cost of covering many languages in one model is sometimes called the curse of multilinguality (Conneau et al., 2019): with fixed capacity, adding languages eventually lowers per-language quality. That is exactly what Laya's numbers show on English.

Laya combines two approaches: a strong English model and a multilingual encoder model, with a router in front that sends each request to the one that can read it.

The router takes the better column in every row. The English checkpoint keeps its lead on English, and every other language goes to mmBERT-base, which was pretrained on text in more than 1,800 languages. See mmBERT.

Routing is cheap: the model card says script detection takes under 0.5 ms in pure Python, before the forward pass. The multilingual checkpoint is also faster, at 32.8 ms for one question against 39.5 ms for the English one on a T4.

Non-Latin scripts are easy to detect exactly: count characters by Unicode block. Latin-script languages are the hard case, because French, Portuguese, Romanian and English share an alphabet. Laya's router uses stopword lists and a diacritic rate, with several safeguards learned from real traffic:

• Accent-stripped text. Mail clients and ticket systems often strip accents. The router's Romance-language lists include unaccented function words, and the model card reports that on accent-stripped MASSIVE, Italian utterances of six or more words routed correctly went from 39% to 80% with that change, with no English prose moved across 20,000 English texts.
• Shared words do not name a language. Words like la, e and que belong to several languages. They count as evidence of "not English" but cannot on their own decide which language.
• Identifiers are not words. URLs, email addresses and dotted names are removed before counting, because fragments like com collide with real Portuguese words.
• Undecided is not English. Very short messages ("Quero cancelar") may carry no identifying words. The router then uses its default, which is English, unless non-English letters tip it. For mostly non-English traffic, pinning the multilingual checkpoint is safer.

Details are in language routing.

• Keep questions in one language. Instructions and option descriptions can stay in English; only the state needs to be in the user's language. Stable option keys keep downstream code language-independent.
• Evaluate per language. A single averaged accuracy hides the languages that fail. Report accuracy and calibration per language on a labelled sample.
• Calibrate per checkpoint. The multilingual checkpoint ships with mean ECE 0.314 (0.106 after temperature refit), the English one with 0.466 (0.081 after refit). Calibration fitted on English traffic does not transfer automatically to other languages.
• Watch the six. Even routed, 6 of 51 tested languages remain below three times random. If you serve low-resource languages, test before automating.
• Mind token budgets. Non-English text often needs more tokens per word. The multilingual checkpoint has a 1,024-token context against 512 for English, which helps.

Send the text as it arrived. Laya Studio routes it and tells you why:

curl -s https://api.laya.studio/v1/systemone \
  -H "Authorization: Bearer $LAYA_STUDIO_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "state": {"message": "Voce pode me mandar a nota fiscal?"},
    "questions": {
      "intent": {
        "type": "choice",
        "instructions": "What does the customer want in message?",
        "criteria": {
          "refund": "money returned or a duplicate charge reversed",
          "technical_help": "a bug, outage or integration problem",
          "billing_question": "a question about an invoice, plan or payment method",
          "cancellation": "wants to cancel or downgrade",
          "other": "none of the other options fits"
        }
      }
    }
  }'

The routing block in the response comes from the same router as the open-source package. For this accent-stripped Brazilian Portuguese message it is:

{
  "model": "multilingual",
  "repo": "convaiinnovations/laya/multilingual",
  "reason": "Latin script but language looks like 'pt', not English",
  "detection": {
    "script": "latin",
    "script_profile": { "latin": 1.0 },
    "language": "pt",
    "is_english": false,
    "language_undecided": false,
    "diacritic_rate": 0.0,
    "non_latin_fraction": 0.0
  },
  "workflow": null
}

The answer itself has the usual choice, probabilities and confidence fields. Billing is per input token (1 credit = 1 input token) whatever the language. Get a key or see the docs for the model override.