AG News

AG News is a text classification dataset built from news articles collected by the AG's corpus project. The commonly used version, introduced by Zhang, Zhao and LeCun (2015), has four balanced classes: World, Sports, Business and Sci/Tech, with 120,000 training and 7,600 test examples, each a title plus a short description.

It is a standard, fairly easy benchmark. Supervised models trained on its training set score well above 90%, so it mainly tests whether a zero-shot or instruction-following classifier understands topic labels at all, and whether it can be steered by label names alone.

Laya on AG News

AG News maps to a single four-option choice question:

{"topic": {"type": "choice",
  "instructions": "What is the news article in `text` about?",
  "criteria": ["World", "Sports", "Business", "Sci/Tech"]}}

The Laya model card reports 0.950 accuracy on AG News with routing, against 0.910 published for TypeSafe Jev 1.13.0. Jev figures are third-party published and were not measured by Laya's maintainers; sample sizes and prompts differ, so read the gap as indicative rather than exact.

What it does and does not tell you

AG News shows that Laya handles low-cardinality topic classification over short English text well. It does not tell you about:

• many-label problems, where Laya is weaker (see Banking77)
• non-English input (see MASSIVE and XNLI)
• domain-specific decisions such as invoice disposition, where the base checkpoints need fine-tuning

Always evaluate on a sample of your own traffic. See Laya vs LLM classifiers and email routing, which is structurally similar.