Choice, score and noul: three primitives for typed decisions

Choice, score and noul are the three question types in the /v1/systemone protocol that Laya uses. A choice question picks one label from options you define, a score question places the input on an ordered scale and returns the expected level, and a noul question returns the probability that a yes/no statement is true.

Look at the decisions a support desk, a moderation queue or an AI agent makes all day and a small number of shapes keep recurring:

• Pick one of several labels. Route to billing, technical or sales. Choose a tool. Assign an intent.
• Place something on an ordered scale. Urgency from low to critical. Severity from none to severe. Frustration from calm to angry.
• Estimate whether a statement holds. Is this phishing? Did the user ask for a refund? Does the message contain personal data?

Statistically these are multi-class classification, ordinal regression and binary probability estimation. The Jev /v1/systemone protocol names them choice, score and noul, and Laya implements the same three. Using one request format for all three matters in practice: a ticket usually needs several of them at once, and they should share one read of the input.

A choice question has a map of labels to descriptions (or a plain list of labels):

{
  "type": "choice",
  "instructions": "What does the customer want in message?",
  "criteria": {
    "refund": "money returned or a duplicate charge reversed",
    "technical_help": "a bug, outage or integration problem",
    "cancellation": "wants to cancel or downgrade",
    "other": "none of the other options fits"
  }
}

Laya renders each option as label: description, places a [MASK] marker before each one, and reads one logit per marker. The softmax over those logits is returned as probabilities, the argmax as choice, and a normalised-entropy confidence:

confidence = 1 − H(p) / log k      where H(p) = −Σ p_i log p_i and k = number of options

Note that this is stricter than the top probability. A four-way answer with probabilities 0.91 / 0.04 / 0.03 / 0.02 has a top probability of 0.91 but a confidence of about 0.71, because the leftover mass is still entropy. Pick thresholds on the field you actually gate on.

Where choice is weak: high cardinality. All options share one budget of 192 tokens (English) or 256 tokens (multilingual). With dozens of options each label gets only a few tokens, and the model card reports 0.425 on 77-label Banking77, against 0.870 published for Jev on 72 labels. Keep sets under about 20 options, or split coarse-to-fine.

A score question takes an ordered list of level descriptions, index 0 first:

{
  "type": "score",
  "instructions": "How frustrated does the customer sound?",
  "criteria": ["calm and neutral", "concerned but civil", "clearly annoyed", "very angry or using strong language"]
}

Laya scores each level at its own marker, like a choice, and returns three things: the distribution over levels, a legend mapping indices to your descriptions, and score, the expected level:

score = Σ i · p(i)

With probabilities 0.05 / 0.22 / 0.58 / 0.14 the score is about 1.82: mostly "clearly annoyed", pulled slightly down by the "concerned" mass. The expected value is useful for sorting and for SLA timers, because it moves smoothly as evidence accumulates. If you need a discrete level, take the argmax of probabilities rather than rounding score, since a bimodal distribution can have an expected value that sits on a level with little mass.

Order matters for training too. For score questions Laya's reward adds a ranked probability score (RPS), which compares cumulative distributions and so penalises a prediction of "severe" more than "clear violation" when the truth is "mild". Plain log loss treats all wrong levels alike. See proper scoring rules.

Where score is weak: the card calls it "the weakest primitive", reporting 0.372 on five-class SST-5 sentiment. Use few, clearly separated levels with concrete descriptions.

noul is a binary question whose answer is a probability rather than a boolean:

{
  "type": "noul",
  "instructions": "Is this email a phishing or scam attempt?",
  "criteria": { "true": "phishing, scam, or fraud", "false": "a legitimate email" }
}

Internally it is always a two-option question in the order false, true. If you omit criteria, Laya uses default descriptions ("no, the statement does not hold" / "yes, the statement holds"). The response carries noul, the probability of true, and a confidence defined for the binary case as:

confidence = max(p, 1 − p)

A noul is the natural input to a threshold. Because it is a probability, you can choose the operating point from your own costs: flag for review at 0.5, auto-block at 0.95, and set both after calibrating on your data.

Where noul is weak: the model card documents that noul "can follow its option labels instead of the state, most strongly on the English checkpoint", returning a confident "no" for clearly positive input (issue #156). The recommended workaround is to ask the same question as a two-option choice with neutral keys:

{
  "type": "choice",
  "instructions": "Is this review positive?",
  "criteria": { "A": "yes, the review is positive", "B": "no, the review is negative" }
}

Some rules of thumb:

• If the answer is naturally yes/no but you have seen noul stick on your data, use a two-option choice.
• If the labels have an order that matters for cost (low < medium < high), prefer score so the training signal and the expected value respect it.
• If a "none of these" outcome is possible, add it explicitly as a choice option. A softmax must put its mass somewhere.
• Do not encode several independent flags as one multi-label choice. Ask separate noul questions. They run in the same forward pass.

The primitives can be mixed freely. Each is answered from the same state in one batched forward pass, and on Laya Studio each is billed the input tokens it reads.

curl -s https://api.laya.studio/v1/systemone \
  -H "Authorization: Bearer $LAYA_STUDIO_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "state": { "message": "Third time asking. The export button still crashes and I need the report for my board meeting tomorrow." },
    "questions": {
      "intent": {
        "type": "choice",
        "instructions": "What does the customer want in message?",
        "criteria": {
          "refund": "money returned or a duplicate charge reversed",
          "technical_help": "a bug, outage or integration problem",
          "billing_question": "a question about an invoice, plan or payment method",
          "information": "general information, pricing or how-to",
          "cancellation": "wants to cancel or downgrade",
          "other": "none of the other options fits"
        }
      },
      "frustration": {
        "type": "score",
        "instructions": "How frustrated does the customer sound in message?",
        "criteria": ["calm and neutral", "concerned but civil", "clearly annoyed", "very angry or using strong language"]
      },
      "is_urgent": {
        "type": "noul",
        "instructions": "Does message communicate time pressure or a deadline?"
      }
    }
  }'

An illustrative response (numbers are examples, not measurements; routing is abbreviated):

{
  "model": "laya-rl-agent",
  "answers": {
    "intent": {
      "type": "choice",
      "choice": "technical_help",
      "probabilities": { "refund": 0.0412, "technical_help": 0.8634, "billing_question": 0.0301, "information": 0.0226, "cancellation": 0.0189, "other": 0.0238 },
      "confidence": 0.6577,
      "action": { "act_probability": 0.9989 }
    },
    "frustration": {
      "type": "score",
      "score": 1.8177,
      "legend": { "0": "calm and neutral", "1": "concerned but civil", "2": "clearly annoyed", "3": "very angry or using strong language" },
      "probabilities": { "0": 0.0521, "1": 0.2214, "2": 0.5832, "3": 0.1433 },
      "confidence": 0.2205,
      "action": { "act_probability": 0.9978 }
    },
    "is_urgent": { "type": "noul", "noul": 0.9046, "confidence": 0.9046, "action": { "act_probability": 0.9992 } }
  },
  "usage": { "input_tokens": 268, "output_tokens": 0 },
  "routing": { "model": "english", "reason": "English Latin text" }
}

It is billed its input tokens: each of the three questions reads the state once. Read the docs for validation rules (a score needs a list, a choice needs at least one option), and sign up for a key with 5 free runs.