Security alert triage with calibrated decisions

Security alert triage is the first-pass review of every alert from your SIEM, EDR or identity provider: is it real, does it involve credentials, how severe is it and what should happen next. Laya answers those questions in one call with probabilities, so confidently benign alerts are closed and uncertain ones reach an analyst.

A security operations centre receives alerts from identity providers, EDR agents, email gateways, cloud audit logs and custom detections. Most are benign: a VPN egress that looks like impossible travel, an admin running a legitimate script, a user who fat-fingered a password. The first-pass question for each alert is the same: is this real, is it about credentials, how bad would it be, and what do we do now?

Tier-1 analysts answer those questions by reading the alert and a handful of context fields. It is repetitive, high-volume and time-critical, and it is the step where alert fatigue causes real incidents to be closed as noise. SOAR playbooks automate the enrichment but usually leave the judgement to a person, or encode it as brittle rules.

The requirements for automating the judgement are strict. Decisions must be fast enough to run on every alert, must come with an honest measure of uncertainty so that anything unclear still reaches a human, and must not invent facts. A model that confidently writes a plausible explanation for an alert it misread is worse than no model.

Laya's laya-typed-decisions checkpoint (ModernBERT-large, 421M parameters, 1,024-token context) was fine-tuned on four synthetic typed-decision workflows, including security_incidents. On the model card's typed-decisions benchmark it scores 0.766 on security incidents (0.766 overall across all four workflows, against a published 0.727 for TypeSafe Jev 1.13.0 and a 0.735 teacher self-agreement ceiling). By primitive, it reaches 0.857 on noul, 0.733 on choice and 0.723 on score questions.

For triage specifically:

• No generated narrative. Laya returns labels and probabilities. It cannot hallucinate an IP reputation or a MITRE technique. It can still be wrong, which is what thresholds are for.
• Calibrated probabilities. A true_positive of 0.15 and one of 0.55 lead to different queues. Laya is trained with RLCD against proper scoring rules so its probabilities are meant to be read literally; refit temperatures on your data to make them so.
• Latency. All five questions share one forward pass. The model card reports 84.5 ms for five questions on a T4 for the English checkpoint, which shares the ModernBERT-large encoder (no separate figure is published for typed-decisions), plus network time to the hosted API. That fits inline in a SOAR playbook.
• Cost and control. Billed per input token, 30% below Jev's list price (see pricing). The weights are Apache-2.0, so teams that cannot send alert data to a hosted API can self-host the same model.

The base checkpoints are not the tool here: on the same benchmark the base English checkpoint scores 0.362, below the 0.461 majority baseline. The capability comes from fine-tuning.

The security-incidents workflow is identified by an exact set of question ids: credential_compromise, disposition, severity, true_positive and urgency. Use exactly those ids, and pass "model": "typed-decisions" in the request so the fine-tuned checkpoint answers. The router never picks that checkpoint silently; the self-hosted Router only auto-detects the workflow when auto_task_detection is enabled.

The model card and package publish the workflow and ids, not the instructions and criteria text used to generate the synthetic training data. The criteria in the example are illustrative. Your SOC's language and alert formats will differ from the synthetic data, so evaluate on labelled historical alerts first.

Design notes:

• Enrich before you ask. Put the alert and the context an analyst would look at in the state: user role, known VPN ranges, recent password resets, MFA push counts. The example includes the VPN egress range that explains the Singapore login. The model can only use what it sees.
• Compute what is computable. Whether an IP falls inside a CIDR range is a lookup, not a judgement. Do it in code and add a boolean such as "second_ip_in_known_vpn": true.
• Keep disposition actionable and include a benign option. Four or five options is plenty.
• severity describes impact if real, independent of whether it is real. Keeping the two separate lets you escalate low-probability, high-severity alerts.
• Stay within context. About 768 tokens of state. Send the alert and top context, not the raw log stream.

Guarding the LLM tools in your SOC

If your SOC also uses LLM assistants over alert data, the guard_questions() preset in the package (jailbreak, prompt_injection, sensitive_data, harm_severity, topic) is a separate, general-checkpoint schema for screening prompts. Alert payloads can carry attacker-controlled text, so checking for prompt injection before an LLM reads them is worthwhile. Run it as a separate request.

The cost structure in security is extreme: closing a real compromise as benign is far more expensive than an analyst spending five minutes on a false positive. Automate only the confident benign end, and use probabilities to prioritise everything else.

Gate on confidence and the raw noul probabilities. Do not use action.act_probability yet: the model card and issue #185 report that it reads close to 1.0 for almost every input, with raw logits running against correctness (AUROC 0.30 on 396 labelled decisions; confidence reached 0.77). Log it for later.

Calibrate on your alerts. The typed-decisions checkpoint reports an ECE of 0.213 on its own benchmark. Fit temperatures per (question type, option count) on a few hundred labelled alerts and check with a reliability diagram before setting thresholds.

A starting policy to tune:

def soc_policy(a):
    if a["credential_compromise"]["noul"] >= 0.5 or a["disposition"]["choice"] == "contain":
        return "page_oncall"
    if a["true_positive"]["noul"] >= 0.4 or a["severity"]["score"] >= 2:
        return "analyst_priority"
    d = a["disposition"]
    if d["choice"] == "close_benign" and d["confidence"] >= 0.8 and a["true_positive"]["noul"] <= 0.05:
        return "auto_close_sampled"
    return "analyst_standard"

Audit a random sample of auto-closed alerts every week and feed misses back into your labelled set. See escalation and human-in-the-loop.

curl -s https://api.laya.studio/v1/systemone \
  -H "Authorization: Bearer $LAYA_API_KEY" \
  -H "Content-Type: application/json" \
  -d @alert_request.json   # {"model": "typed-decisions", "state": {...}, "questions": {...}}

import os, requests

SEC_IDS = {"credential_compromise", "disposition", "severity", "true_positive", "urgency"}

def triage_alert(state: dict, questions: dict) -> dict:
    assert set(questions) == SEC_IDS
    try:
        r = requests.post(
            "https://api.laya.studio/v1/systemone",
            headers={"Authorization": "Bearer " + os.environ["LAYA_API_KEY"]},
            json={"model": "typed-decisions", "state": state, "questions": questions},
            timeout=5,
        )
        r.raise_for_status()
        return r.json()["answers"]
    except requests.RequestException:
        return {}  # fail open to the analyst queue, never to auto-close

export async function triageAlert(state: object, questions: object) {
  try {
    const res = await fetch("https://api.laya.studio/v1/systemone", {
      method: "POST",
      headers: { Authorization: "Bearer " + process.env.LAYA_API_KEY, "Content-Type": "application/json" },
      body: JSON.stringify({ model: "typed-decisions", state, questions }),
      signal: AbortSignal.timeout(5000),
    });
    if (!res.ok) return null; // route to analyst queue
    return (await res.json()).answers;
  } catch {
    return null;
  }
}

The important integration detail is the failure mode: if the call fails or times out, the alert goes to a human, never to auto-close. Store the full response with the case for audit. If policy forbids sending alert data to a third party, the Apache-2.0 weights can be self-hosted with the same protocol. Get a key or read the docs.

Alert payloads contain usernames, IP addresses and hostnames. Laya Studio's primary inference pool runs on dedicated GPUs located in Switzerland, and every API response says where it was processed in the x-laya-region header. The text and questions you send are processed in memory and discarded when the answer is returned: they are never written to a database or log, and never used to train anything.

Turn on Swiss-only mode for a workspace (or send the x-laya-residency: ch header on a request) and requests are only ever answered in Switzerland. If the Swiss pool is unavailable you get an error, never a silent detour abroad. For billing and debugging, only request metadata (time, status, number of questions, latency) is kept, for 30 days. All traffic uses TLS, and the GPU servers accept requests only from the Laya Studio gateway. If alert data must never leave your own network, self-host the Apache-2.0 weights instead. You remain responsible for your legal basis to process personal data. Details: Swiss data residency.

• Synthetic training data. The 0.766 security-incidents figure is on a synthetic benchmark's test split. Real alerts from your tools will look different. Measure on your own labelled history.
• Criteria wording is yours. The training criteria are not published; wording affects accuracy. Test variations.
• It sees only the state. No threat intelligence, no log search, no memory of related alerts unless you include them. Correlation is your SIEM's job.
• Adversarial input. Alerts can contain attacker-controlled strings (user agents, email subjects, file names). A classifier cannot be instructed the way an LLM can, but crafted text can still shift probabilities. Keep auto-close thresholds strict.
• Score questions are weaker (0.723 by primitive) than noul (0.857). Use severity and urgency for ordering, not as sole gates.
• Context is about 768 tokens of state per question.
• Accountability stays with the SOC. Treat Laya as a tier-0 filter and prioritiser, with humans owning containment decisions.