Softmax

The softmax function maps a vector of real numbers (the logits) to a probability distribution:

softmax(z)_i = exp(z_i) / Σ_j exp(z_j)

Every output is positive and they sum to 1. Softmax is monotonic, so the largest logit always gets the largest probability, and it is invariant to adding a constant to every logit. Implementations subtract the maximum logit first for numerical stability, which changes nothing mathematically.

Softmax outputs look like probabilities, but nothing about the function makes them calibrated. A network trained with cross-entropy can push logits far apart and produce 0.99 on inputs it gets wrong half the time. That is what temperature scaling corrects: it divides the logits by T before softmax.

Softmax in Laya

In Laya the softmax is taken over a question's own options, not over a fixed label vocabulary. Each option is introduced by a [MASK] token; the decision head reads the hidden state at each of these option markers and produces one logit per option. The runtime divides those logits by the temperature for the question's bucket and applies a numerically stable softmax (exp(z − max z), normalised). The result is:

• the probabilities map for a choice question,
• the per-level probabilities of a score question, from which the expected level is computed,
• the probability of "true" (noul) for a noul question.

Because the answer space is whatever options you send, a request with three options gets a three-way softmax and one with nine gets a nine-way softmax, with no retraining. Confidence is then derived from that distribution; see entropy-based confidence and option-marker scoring.