Act or escalate: routing decisions by calibrated confidence

Act-or-escalate routing lets an automated system act on a decision only when its confidence is above a threshold, and hands everything else to a human, a larger model or a safe default. The threshold comes from the relative cost of a wrong action and an unnecessary escalation, and it only works if confidence scores are calibrated.

When a model makes an automated decision, it can fail in two ways. It can act and be wrong: route a legal threat to the sales queue, auto-approve a fraudulent invoice. Or it can escalate when it did not need to: send an obvious password-reset request to a human, burning time and money. Every automated decision system trades these off, whether its designers chose the trade-off or not.

The formal name for this is selective prediction (or classification with a reject option), studied since Chow (1970). The model is allowed to abstain. Two numbers describe how well it does that:

• Coverage: the fraction of inputs the model acts on.
• Selective risk: the error rate on the inputs it acts on.

Lowering the threshold raises coverage and usually raises risk. Plotting one against the other gives a risk-coverage curve, and a good confidence signal is one whose curve stays low as coverage grows (Geifman and El-Yaniv, 2017).

If probabilities are calibrated, the threshold can be derived rather than guessed. Say acting wrongly costs C_err, escalating costs C_esc, and acting correctly costs nothing. With probability p that the model is right, acting has expected cost (1 − p) · C_err. Act when that is below the cost of escalating:

act if  (1 − p) · C_err < C_esc
   i.e. p > 1 − C_esc / C_err

Examples:

The last row shows why calibration is not optional. A threshold of 0.99 is only meaningful if "0.99" is actually right 99% of the time. An over-confident model reports 0.99 far more often than it is right at that rate, and the gate lets errors through. See calibrated probabilities and temperature scaling.

In practice the costs are rough, so treat the formula as a starting point and tune the threshold on a labelled validation set against the coverage you can afford to escalate.

Not every number a model outputs is a good gate. The useful property is discrimination: are correct answers ranked above incorrect ones? This is measured with AUROC, where 0.5 is no better than chance and 1.0 is perfect separation. Hendrycks and Gimpel (2016) showed the plain maximum softmax probability is already a reasonable baseline for detecting misclassified examples.

Two warnings:

1. Discrimination and calibration are different. A signal can rank errors well (good AUROC) and still be over-confident in absolute terms (bad ECE). You need ranking to choose which cases to escalate and calibration to choose how many.
2. Confidence cannot detect inputs the model cannot read. The Laya model card reports the English checkpoint at 0.000 accuracy with 0.952 confidence on Khmer. No threshold on that confidence would catch it. Out-of-distribution protection has to come from upstream, which for Laya is language routing.

Laya's decision model includes an act head, designed exactly for this job. It reads the pooled [CLS] representation plus four features of the answer distribution: the top probability, the margin between the top two options, the normalised entropy, and the option count. Its output is returned in every answer as action.act_probability.

The model card is direct: "action.act_probability carries no usable signal yet. It reads 1.0 for almost every input, and its raw logits run against correctness (AUROC 0.30 on 396 labelled decisions). Gate on confidence instead, which reaches an AUROC of 0.77 on the same items."

An AUROC of 0.30 is worse than random: in that evaluation the act head was more likely to favour acting on wrong answers than on right ones. Until a checkpoint fixes this (tracked as issue #185), treat act_probability as a reserved field.

Laya returns two usable signals per answer:

A workable procedure:

1. Route first. Make sure the input reaches a checkpoint that can read it. Laya Studio does this automatically.
2. Collect a labelled sample of a few hundred items per question, drawn from real traffic.
3. Calibrate. Fit a temperature per question type and option-count bucket if probabilities are over-confident.
4. Plot accuracy against coverage as you vary the confidence threshold.
5. Pick the threshold that meets your error budget, and check the coverage is affordable.
6. Monitor. Log the confidence distribution in production; a shift usually signals new kinds of input.

Remember that entropy-based confidence is stricter than the top probability. A four-way answer at 0.91 top probability has a confidence of about 0.71, so a threshold that looks conservative on one field is aggressive on the other. Choose thresholds on the field you gate.

Escalation does not have to mean a human. Common targets:

• A second, slower model. A large LLM reasons over the uncertain minority. This is the System 1 / System 2 pattern: fast intuition for the bulk, slow deliberation for the hard cases.
• A clarifying question. In a chat agent, ask the user rather than guess.
• A human queue, prioritised by a score question such as urgency.
• A safe default. For guardrails, blocking or sandboxing when uncertain may be the right escalation.

curl -s https://api.laya.studio/v1/systemone \
  -H "Authorization: Bearer $LAYA_STUDIO_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "state": {"alert": "Impossible travel: login from Lagos 40 minutes after login from Oslo for user j.doe. MFA passed."},
    "questions": {
      "disposition": {
        "type": "choice",
        "instructions": "What should happen with alert?",
        "criteria": {
          "close": "benign or expected activity",
          "investigate": "needs an analyst to look",
          "contain": "likely compromise, lock the account now"
        }
      },
      "credential_compromise": {"type": "noul", "instructions": "Does alert indicate stolen credentials?"}
    }
  }'

Then gate in your code. The thresholds below are placeholders to be replaced with values tuned on your own labelled data:

ans = resp["answers"]
disp = ans["disposition"]

AUTO = {"close": 0.80, "investigate": 0.0, "contain": 0.60}   # tune on your data

if disp["choice"] == "investigate" or disp["confidence"] < AUTO[disp["choice"]]:
    escalate(resp, reason="low confidence or needs analyst")
else:
    act(disp["choice"])

# ignore ans[...]["action"]["act_probability"] until the model card says otherwise

Two questions read the state twice, billed per input token. See the docs for response fields and /signup for a free-tier key.