Log score

The log score (logarithmic scoring rule) rewards a probabilistic forecast with the natural log of the probability it assigned to the outcome that occurred: S(p, y) = log p_y. Its negative is the familiar log loss or cross-entropy used to train most classifiers.

It is a strictly proper scoring rule and the only proper rule (up to affine transforms) that depends solely on the probability of the realised outcome, which gives it an information-theoretic reading: the expected log score is the negative cross-entropy between the true distribution and the forecast.

Its signature property is how it treats confident mistakes. Assigning 0.01 to the true answer costs log 0.01 = -4.6; assigning 0 costs negative infinity. That makes it an excellent pressure against over-confidence and a poor fit for noisy labels, because one mislabelled example can dominate a batch.

The log score in Laya

Laya's RLCD reward uses the log score as its main term, combined with the spherical score (weight 0.5) and, for ordinal questions, the ranked probability score. The package floors the log-probability at -9.21 (roughly ln 0.0001) so a single near-zero probability cannot produce an unbounded penalty and destabilise the policy-gradient update.

This matters to buyers because it explains a failure seen in other systems. On DAIR Emotion, the model card notes that TypeSafe Jev assigned zero probability to the true label on 16% of examples. Under a log score, a zero on the true label is the worst possible outcome, so a model trained on it is pushed to keep some mass on every plausible option.

See proper scoring rules and calibrated probabilities.