Skip to content

How-to

Implementing Explainable AI (XAI) for System 1 Decision Models: A Developer's Guide

A developer's guide to explainable AI for System 1 models, using calibrated probabilities, perturbation audits, and auditable decision APIs.

Laya Studio7 min read

On this page (8 sections)

You make a System 1 decision model explainable by returning a calibrated probability for every question and publishing the threshold that turned that probability into a yes, no, or score. When the arXiv audit of Jev-style System 1 models was submitted on 9 Nov 2026, its central question was whether these models’ confidence and decisions can be trusted (arXiv:2611.12521). My answer is yes, but only if you treat the probability as part of the contract, not an internal detail.

Key takeaways

  • System 1 models return a typed decision in one forward pass, so there is no generated text to audit—only the score and the rule (explainx.ai).
  • A calibrated probability means the number reflects how often the model is right; 0.82 should be right about 82% of the time on similar cases.
  • Explainability comes from logging the probability, the threshold, and the reason code together.
  • Run perturbation tests before you ship; if changing an input barely moves the probability, your model is ignoring that signal.
  • Laya runs in Switzerland with zero data retention and is priced per question (100,000 free, then $5 per million).

1. Start with a System 1 model, not a generative model

A System 1 model skips text generation and returns a typed decision in one parallel pass (explainx.ai). For explainability, that matters because there is no free-text rationale to hallucinate, redact, or misread. You get a choice, a score, or a yes/no, plus a probability you can audit.

I learned this the hard way when I tried to explain a support-triage decision made by an LLM. The model wrote a polite paragraph about “escalating due to urgency,” but the actual trigger was a keyword it buried in the middle of the text. With a System 1 model, the trigger is explicit: the input goes in, a typed decision comes out.

Laya is an open-source (Apache-2.0) System 1 decision model hosted in Switzerland. It returns results in about 120 ms from Europe, and it is compatible with TypeSafe Jev’s /v1/systemone protocol. If you are already using Jev, you can switch without rewriting your request schema (How to Switch from TypeSafe Jev to Laya Studio).

AttributeSystem 1 decision APILLM classification prompt
OutputTyped decision + probabilityFree text
Latency~120 ms from EuropeVariable, often seconds
Hallucination riskNo text generated, so no fabricated labelsHigh
Audit trailStructured log of probability and thresholdParsing required
Cost modelPer questionPer token

2. Request calibrated probabilities for every question

2. Request calibrated probabilities for every question

Calibrated probability is the number that reflects how often the model is right. When your decision API returns 0.82 for “escalate,” it should be correct about 82% of the time across similar cases. That turns a black-box label into a measurable claim you can test.

Laya returns a calibrated probability for every question in a single forward pass. You send any text or JSON plus typed questions—choice, score, or yes/no—and get back one probability per question. No text is generated, so it cannot hallucinate a label.

Example request:

json
POST https://api.laya.studio/v1/systemone
{
 "input": {
 "subject": "Billing error on invoice #4421",
 "body": "I was charged twice and cannot reach support.",
 "tier": "enterprise"
 },
 "questions": [
 { "id": "escalate", "type": "yes_no" },
 { "id": "priority", "type": "choice", "options": ["low", "normal", "high"] }
 ]
}

Example response:

json
{
 "escalate": { "answer": true, "probability": 0.87 },
 "priority": { "answer": "high", "probability": 0.91 }
}

The first time I exposed a probability to a customer, they asked what 0.87 meant. I had to explain that it is not a confidence score in the colloquial sense; it is a calibrated estimate of accuracy. If you cannot explain that distinction, you are not ready to put probabilities in front of users.

3. Turn probabilities into explicit decision rules

3. Turn probabilities into explicit decision rules

A probability becomes an explanation only when you attach a threshold and a reason. I do not ship “probability: 0.64” to a user; I ship “escalate: true, because escalation probability 0.64 exceeds the 0.60 threshold for senior review.”

Use this checklist when you define your bands:

  • Pick a threshold per question based on the cost of a wrong answer
  • Assign a reason code to each band (auto-approve, review, reject)
  • Log the exact probability and threshold used for every decision
  • Flag near-threshold decisions (within 0.05 of the cutoff) for human review

Thresholds are business decisions, not model truths. If you move the escalation threshold from 0.60 to 0.80, the explainability does not change, but your false-positive rate will. Be honest about that trade-off in your documentation.

4. Audit the model with perturbation tests

I do not trust a calibrated probability until I have nudged the input and watched the probability move. The arXiv audit of Jev-style System 1 models uses explainability and perturbation to test whether confidence tracks the decision (arXiv:2611.12521). If you change “urgent” to “low priority” and the probability stays at 0.90, your explainability story is broken.

Run a small perturbation suite before each release:

Input variantEscalate probabilityVerdict
Original ticket0.78Baseline
“urgent” removed0.31Sensitive to urgency
Customer tier changed to free0.79Ignores tier (red flag)
Synonym swapped (“invoice” → “bill”)0.76Robust to wording

If a variant changes the probability without changing the meaning, your model is brittle. If a variant changes the meaning but not the probability, your model is ignoring a signal. Both are explainability failures.

5. Log decisions so they are auditable later

Explainability is retrospective as much as it is real-time. If a regulator or a customer asks why a decision was made in June, you need the probability, the threshold, and the model version—not a screenshot.

Log schema I use:

FieldTypeExample
input_hashstringsha256("billing error...")
question_idstringescalate
answerboolean/stringtrue
probabilityfloat0.87
thresholdfloat0.60
reason_codestringescalate_high_prob
model_versionstringlaya-systemone-v1
timestampISO-86012026-11-20T14:32:01Z
latency_msinteger118

Laya itself has zero data retention, which means it does not store your inputs or decisions (The Swiss Data Advantage). That is good for privacy, but it means the audit log is your responsibility. Store it where your compliance team can reach it.

6. Ship it through a decision API

The final step is exposing the model as an interpretable decision API so your application receives structured JSON, not a chat transcript. This is what makes it an interpretable AI decision API rather than a black-box classifier.

Python example:

python
import os
import requests

LAYA_API_URL = os.getenv("LAYA_API_URL", "https://api.laya.studio/v1/systemone")
API_KEY = os.getenv("LAYA_API_KEY")

def decide(ticket):
 payload = {
 "input": ticket,
 "questions": [
 {"id": "escalate", "type": "yes_no"},
 {"id": "priority", "type": "choice", "options": ["low", "normal", "high"]}
 ]
 }
 r = requests.post(LAYA_API_URL, json=payload, headers={"Authorization": f"Bearer {API_KEY}"})
 r.raise_for_status()
 result = r.json()

 escalate = result["escalate"]
 if escalate["probability"] < 0.50:
 return {"action": "auto_resolve", "reason": "low_escalation_probability"}
 if escalate["probability"] < 0.80:
 return {"action": "queue_for_review", "reason": "near_threshold", "probability": escalate["probability"]}
 return {"action": "escalate", "reason": "high_escalation_probability", "probability": escalate["probability"]}

If you are comparing vendors, check the pricing page for the exact per-question math (AI Decision API Pricing). I prefer per-question pricing because I can budget classification precisely, and I never pay for tokens I do not use.

7. Keep the human in the loop

Automated explainability does not absolve you of ethical responsibility. A calibrated probability is still a statistical guess, and near-threshold cases deserve human judgment. I review a sample of decisions weekly, especially anything within 0.05 of a threshold. If the model is consistently wrong on a cohort—say, non-native English speakers—I retrain or adjust the rule rather than blaming the probability.

This is where ethical AI development meets operations. Explainability is not a compliance checkbox; it is the mechanism that lets you catch when the model is making decisions for the wrong reasons. For more on how calibrated probabilities work in regulated environments, see my deeper write-up (Beyond 'Black Box': Implementing Explainable AI with Calibrated Probabilities).

If you are already routing tickets, scoring risk, or moderating content in real time, you can try this on Laya Studio with 100,000 free questions per month (laya.studio).

FAQ

What is XAI for real-time AI?

Explainable AI for real-time systems means every low-latency decision comes with a reason you can inspect and audit. In a System 1 model, that reason is the calibrated probability plus the threshold that produced the answer, delivered in milliseconds.

What is an interpretable AI decision API?

It is an endpoint that takes structured input and returns a structured decision—such as a choice, score, or yes/no—along with a probability and reason code. Unlike an LLM chatbot, there is no generated text to parse or second-guess.

How do calibrated probabilities improve explainability?

A calibrated probability tells you how often the model is right at that confidence level. Instead of trusting an unexplained label, you can verify the probability against actual outcomes and adjust thresholds when the model drifts.

How do you audit System 1 AI models?

Run perturbation tests: change one input at a time and confirm the probability moves in a sensible direction. Log every decision with its probability, threshold, and model version so you can reproduce any past call.

Do I need this if I am not in a regulated industry?

Not strictly, but explainability pays off even without regulation. It makes debugging faster, reduces support escalations, and prevents the “why did the AI do that?” conversation from becoming a bottleneck.

Topics

  • explainable AI System 1 models
  • XAI for real-time AI
  • interpretable AI decision APIs
  • calibrated probabilities explainability
  • auditing System 1 AI
  • building transparent AI
  • developer guide XAI
  • ethical AI development

Keep reading

Live demo

Reading is good. Trying is better.

See real answers on five example messages, with a calibrated probability for every answer in about a tenth of a second. Sign up and your first 5 runs on your own messages are free.

Example answer, captured live

Answered in Switzerland

“Hi, I was charged twice for order #4821 ($129.00). Please refund the duplicate charge before Friday, our books close then. This is the second billing mistake this quarter and we're starting to look at other vendors.”

What does the customer want?

  • refund100%
  • other<0.1%
  • cancel0%
Is it urgent: 17.7%Might they leave: 30.3%

0 words generated · 3 questions in one pass · 354 ms round trip when captured