Abstention

Abstention (also called selective prediction or the reject option) means a classifier declines to commit to an answer when it is not sure enough. Instead of acting, the system defers: to a human reviewer, a slower model, or a default safe path.

The classic formulation trades coverage (the fraction of inputs the system answers) against selective risk (the error rate on the inputs it does answer). Raising the confidence threshold lowers coverage and, if the confidence signal is informative, lowers the error rate on what remains. Plotting that trade-off gives a risk–coverage curve.

What abstention requires

Abstention is only as good as the signal it thresholds on. Two properties matter:

1. Ranking: wrong answers should, on average, get lower confidence than right ones. This is measured with AUROC of confidence against correctness.
2. Calibration: a stated 0.9 should be right about 90% of the time, so a threshold means what you think it means. See calibration and expected calibration error.

Abstention with Laya

Every Laya answer carries a confidence field. For choice and score questions it is 1 − H(p)/log k, the entropy-based confidence of the option distribution; for noul it is max(p, 1 − p). On the maintainers' labelled set of 396 decisions, confidence reached an AUROC of 0.77 against correctness, which makes it a usable abstention signal.

Laya also returns action.act_probability from a dedicated act/escalate head, but the model card states it carries no usable signal yet (issue #185): it reads about 1.0 for almost every input. Abstain on confidence, not on act_probability.

Two caveats: the checkpoints ship over-confident, so fit temperature scaling on your own data before choosing thresholds; and confidence cannot rescue a checkpoint reading a language it does not understand, which is why language routing comes first. The act/escalate routing guide shows threshold selection in practice.