Temperature scaling

Temperature scaling is the simplest post-hoc fix for calibration. Before the softmax, every logit is divided by a scalar T:

p_i = exp(z_i / T) / Σ_j exp(z_j / T)

T > 1 flattens the distribution (less confident), T < 1 sharpens it. Because dividing by a positive constant preserves ordering, the predicted class never changes: accuracy is untouched and only the probabilities move. T is fitted on a held-out labelled set, usually by minimising negative log-likelihood. Guo et al. (2017) showed that this single parameter fixes most of the over-confidence of modern neural classifiers.

How Laya applies it

Laya does not fit one global temperature. It keeps one per bucket of (question type, option count), with option-count buckets 2, 3-5, 6-10 and 11+, falling back to a per-type temperature when no bucket exists. So a two-option noul question and a twelve-option choice question are softened differently.

The runtime also refuses dangerous temperatures. Every value is clamped to [0.5, 5.0]. The reason is concrete: the shipped choice:11+ bucket was fitted to 0.1006, which multiplies logits roughly tenfold and would publish a 0.24 top probability as 0.99. The loader warns when a checkpoint contains such values and tells you to treat affected confidences as uncalibrated.

The model card reports that refitting per bucket moves mean ECE from 0.466 to 0.081 (English) and 0.314 to 0.106 (multilingual). Its advice, which we repeat: fit temperatures on your own data before trusting the probabilities. See temperature scaling for a worked example.