CALIBER
Quickstart

Cookbook 03 · Foundations

Policy-Safe Decision Tool

Deterministic business logic first, then approval-gated side effects, then an explanation layer.

FoundationsCore40–60 min
What you build. A refund decision flow: a deterministic eligibility node, a hard approval gate before the (mocked) refund write, and an optional explanation that can never contradict the decision.
Surfaces: Tools Workflows Judges Evaluations
What you learn
  • Register a shipped callable as a tool from the Spec form (no code)
  • Author deterministic logic as a visual decision table
  • See write tools auto-mock in the sandbox + gate in a workflow
  • Drive the human-approval gate in the Run Monitor
  • Score an explanation with an LLM judge — after the deterministic lane is green

Implementation flow

flowchart LR
  RI[/run input JSON/] --> DR[Data Transform: ordered decision table]
  DR --> HA{{Human Approval · gates every run when enabled}}
  HA -->|approve| IR[Tool: initiate_refund · mocked + gated]
  DR -.optional.-> EX[Explanation prompt] -.as a scorer.-> JF[Judge: ExplanationFaithfulness]
  LO[Tool: lookup_order] -.Tools sandbox demo.-> SBX[Fixtures + Hardening]

Step-by-step UI only

Each step shows the exact navigation path, the fields to fill, the button to click, and a snapshot of the CALIBER screen you'll be on.

  1. Library › Tools › Register Tool creates · tool: lookup_order (read) lookup-order.tool.json ↓
    Register the read-only order lookup. Clicking Register Tool opens the tool wizard. A registered Tool must point at importable Python, so you give it a Module Path and Callable Name for a shipped callable — no code to write. Mark it read and allow it in preview so the sandbox runs it live.
    Fill in
    Namelookup_order
    Module Pathcaliber.workflows.demo_tools
    Callable Namelookup_order
    Side effect levelread
    Allow in previewenabled (run the real callable in sandbox)
    Input / Output schemapaste the input_schema (requires order_id) and output_schema from lookup-order.tool.json into the Schema step's raw-JSON fields
    Click Register Tool
    You'll see: The tool lookup_order is registered and opens in its workspace with stage tabs (Spec, Sandbox, Fixtures, Test Runs, Hardening, Publish).
    Screen snapshot
    caliber · Library › Tools › Register Tool
    ToolsRegister Tool
    Register Tool
  2. Library › Tools › Register Tool creates · tool: initiate_refund (write) initiate-refund.tool.json ↓
    Register the refund write tool the same way. Because side_effect_level is write, it is automatically mocked in the sandbox and becomes approval-gated when used in a workflow — exactly the safety behavior this lab wants. Leave Allow-in-preview off (writes are mocked regardless).
    Fill in
    Nameinitiate_refund
    Module Pathcaliber.workflows.demo_tools
    Callable Nameinitiate_refund
    Side effect levelwrite
    Input / Output schemapaste the input_schema (requires order_id) and output_schema from initiate-refund.tool.json
    Click Register Tool
    You'll see: The tool initiate_refund appears in the Tools list and opens in its own workspace.
    Screen snapshot
    caliber · Library › Tools › Register Tool
    ToolsRegister Tool
    Register Tool
  3. Library › Tools › lookup_order › Sandbox
    Run the read tool live in the Sandbox to confirm it imports and returns real output. Because it is a read tool with preview allowed, you get the genuine lookup result.
    Fill in
    Input JSON{"order_id": "ord_1120"}
    Click Test Run
    You'll see: A real response object (e.g. {order_id, items, status}) with no mocked flag — proving the live read works.
    Screen snapshot
    caliber · Library › Tools › lookup_order › Sandbox
    ToolsSandbox
    Test Run
  4. Library › Tools › initiate_refund › Sandbox
    Run the write tool in the Sandbox to confirm it is safely mocked. Write/external-action tools never execute for real in preview, so you can test the contract without issuing an actual refund.
    Fill in
    Input JSON{"order_id": "ord_1120", "amount": 49.0}
    Click Test Run
    You'll see: A response carrying mocked: true, confirming the refund did not really fire in the sandbox.
    Screen snapshot
    caliber · Library › Tools › initiate_refund › Sandbox
    ToolsSandbox
    Test Run
  5. Library › Tools › lookup_order › Fixtures creates · tool fixtures: lookup_order suite refund-fixtures.jsonl ↓
    Save deterministic fixtures (named cases with assertions) so you can score the tool repeatably. Use Add case once per fixture; each case pairs an input JSON with an assertion. lookup_order takes only order_id, so each case's input JSON must be scoped to just that key — passing the full multi-key fixture row would raise a TypeError and fail the case. The available assertion types are no_error, output_contains, and equals — use output_contains with a value like status to assert the lookup output.
    Fill in
    Casesfor each row in refund-fixtures.jsonl, add a case whose input JSON is scoped to just the order id (e.g. {"order_id": "ord_1120"}) paired with the assertion output_contains = status
    Click Save cases
    You'll see: The saved fixture cases appear in the panel, ready to run as a suite.
    Screen snapshot
    caliber · Library › Tools › lookup_order › Fixtures
    ToolsFixtures
    Save cases
  6. Library › Tools › lookup_order › Hardening creates · hardening run: lookup_order pass rate
    Run the saved fixture suite. The deterministic Hardening lane scores the assertions inline and reports a pass rate you can gate on (target ≥ 0.97). Save the cases first if you edited them, then run.
    Click Run calibration
    You'll see: An immediate result line: Pass rate NN% — passed/total — recorded as a run with per-case pass/fail. No waiting for a background job.
    Screen snapshot
    caliber · Library › Tools › lookup_order › Hardening
    ToolsHardening
    Run calibration
  7. Library › Tools › lookup_order › Test Runs creates · pinned baseline: lookup_order hardening
    Pin the passing hardening run as the tool's baseline so future fixture/schema edits are measured against this known-good result. Open the run in the Test Runs history and pin it.
    Click Set as baseline
    You'll see: The run gets a Baseline marker; the Test Runs tab now shows deltas against it. (Optional: edit a fixture or schema and re-run to watch a regression delta appear.)
    Screen snapshot
    caliber · Library › Tools › lookup_order › Test Runs
    ToolsTest Runs
    Set as baseline
  8. Compose › Workflows › New Workflow creates · workflow: refund-decision-flow (hitl_review)
    Create the workflow from the Human Review template (the hitl_review kind) — it ships the human_approval node you need for the approval gate. Name it, then click the template tile to create it. Starting from this template means the gate is already wired for you.
    Fill in
    Namerefund-decision-flow
    Templatethe Human Review tile (Agent → PII redact → human approval → output)
    Click Human Review
    You'll see: The workflow opens in the editor with a starting graph that already contains a human_approval node.
    Screen snapshot
    caliber · Compose › Workflows › New Workflow
    WorkflowsNew Workflow
    Human Review
  9. Compose › Workflows › refund-decision-flow › Editor creates · python_code node: decide_refund decide_refund.py ↓
    Add the deterministic decision as a Python Code node (drag it from the palette, or use the quick-add + menu, then paste the body in the node inspector). Custom decision logic lives in a python_code node (not a registered tool), so it versions with the workflow and needs no registration. This node sits first after START and reads the run input directly: a python_code node receives the run input as an unparsed string, so the pasted run_python_node(...) entrypoint parses it as JSON and reads order_state, risk_flags, amount, and days_since_order from it.
    Fill in
    Python Code node bodypaste the body of decide_refund.py (it defines decide_refund(...) and the run_python_node(...) entrypoint that parses the JSON run input and returns {decision, reason_code, requires_approval})
    Click 💾 Save
    You'll see: A Python Code node for the refund decision is on the canvas with the pasted body saved to the draft.
    Screen snapshot
    caliber · Compose › Workflows › refund-decision-flow › Editor
    WorkflowsEditor
    💾 Save
  10. Compose › Workflows › refund-decision-flow › Editor creates · workflow graph: decide → approve → refund
    Wire the safety chain by adding the nodes and connecting them in order: START → decide_refund (python_code, which parses the JSON run input) → human_approval → initiate_refund. Add the initiate_refund Tool node from the palette, drag port-to-port to connect, and route through the existing human_approval gate so the write only fires after approval. The human_approval node has no condition field: when runtime approvals are enabled on the run, it pauses every run that reaches it (it does not read requires_approval). The requires_approval flag from decide_refund is informational only — surfaced for the reviewer, not consumed by the gate. (Keep lookup_order as the Tools-sandbox demo from the earlier steps; it is not part of this decision chain — decide_refund reads the decision fields straight from the run input.)
    Fill in
    Nodes to adda Tool node for initiate_refund (decide_refund and human_approval are already on the canvas)
    EdgesSTART → decide_refund → human_approval → initiate_refund
    Click 💾 Save
    You'll see: The saved draft graph shows START → decide_refund → human_approval → initiate_refund in sequence, with the approval gate sitting before the refund write.
    Screen snapshot
    caliber · Compose › Workflows › refund-decision-flow › Editor
    WorkflowsEditor
    💾 Save
  11. Compose › Workflows › refund-decision-flow › Run Monitor creates · run evidence: high-risk blocked at approval refund-fixtures.jsonl ↓
    Prove the gate holds. Runtime approvals are a deployment-level setting, not a per-run checkbox: an admin enables them in the environment by setting CALIBER_WORKFLOW_RUN_RUNTIME_APPROVALS_ENABLED=true together with checkpointing (CALIBER_WORKFLOW_RUN_CHECKPOINTING_ENABLED=true) and the run queue (CALIBER_WORKFLOW_RUN_QUEUE_ENABLED=true) in .env, then restarting — the approve/resume routes reject requests unless all three are on. (Their current values are shown read-only under Settings › Workflow runs; only LLM credentials are live-editable in the UI, so these flags come from the deployment env, not a Settings toggle.) With those enabled, open the Run Monitor panel in the editor, paste a case as the run input, and execute. The human_approval node then pauses the run at the gate instead of letting the write fire — it pauses every run that reaches it, regardless of the decision's requires_approval value. Use a high-risk case so the paused run is also one a reviewer would actually hold.
    Fill in
    Run inputa case from refund-fixtures.jsonl, e.g. the high-risk F08 {"order_id":"ord_1166","order_state":"delivered","risk_flags":["fraud_suspected"],"amount":49.0,"days_since_order":4}
    Click Run
    You'll see: The run advances to the gate and stops at status waiting_approval (shown as "Awaiting approval") — the refund has NOT fired, and Approve / Reject buttons appear.
    Screen snapshot
    caliber · Compose › Workflows › refund-decision-flow › Run Monitor
    WorkflowsRun Monitor
    Run
  12. Compose › Workflows › refund-decision-flow › Run Monitor creates · run evidence: approve marks approved, resume fires refund
    Clear the paused run in two steps. First click Approve: this only records the approval decision (the approval row flips to approved) — the run stays at waiting_approval and the write does NOT fire yet. Then click Resume, a separate action, which advances the paused run from the checkpoint; only now does initiate_refund execute. (Resume refuses to advance until an approved decision exists, so the order matters.)
    Click Approve, then Resume
    You'll see: After Approve the status is still waiting_approval with the approval marked approved; after Resume the run advances and completes, and the timeline shows initiate_refund running only AFTER the resume — the write never fires while the run is paused at the gate.
    Screen snapshot
    caliber · Compose › Workflows › refund-decision-flow › Run Monitor
    WorkflowsRun Monitor
    Approve, then Resume
  13. Compose › Workflows › refund-decision-flow › Run Monitor creates · run evidence: rejected → write never fires
    Now prove the gate also stops the write entirely when you reject. Start another run (runtime approvals are still on at the deployment level) so it pauses at the gate, then click Reject instead of Approve. Rejection ends the run as failed — there is no Resume after a rejection — and the refund never fires.
    Fill in
    Run inputany case from refund-fixtures.jsonl, e.g. F08 {"order_id":"ord_1166","order_state":"delivered","risk_flags":["fraud_suspected"],"amount":49.0,"days_since_order":4}
    Click Reject
    You'll see: The run ends at status failed (error approval_rejected) — initiate_refund never ran and the run cannot be resumed, confirming the write is hard-gated behind the approval decision.
    Screen snapshot
    caliber · Compose › Workflows › refund-decision-flow › Run Monitor
    WorkflowsRun Monitor
    Reject
  14. Library › Prompts › New prompt creates · prompt: refund-explanation refund-explanation.md ↓ policy-reason-normalizer.md ↓ customer-safe-refund-language.md ↓
    Add the optional explanation layer that phrases the decision for the customer without changing it. Use the Write / paste on-ramp, name it, and paste the body (the lines below the YAML frontmatter).
    Fill in
    Prompt namerefund-explanation
    Prompt textpaste the body of refund-explanation.md (below the frontmatter; variables decision, reason_code, order_state, customer_name)
    Commit messagev1 explanation layer — explains, never overrides, the deterministic decision
    Click Create
    You'll see: The prompt refund-explanation is created at v1 and opens in its workspace, ready to test. (Optionally also create the two support skills below to govern its wording.)
    Screen snapshot
    caliber · Library › Prompts › New prompt
    PromptsNew prompt
    Create
  15. Library › Prompts › refund-explanation › Test Sets creates · generated cases for refund-explanation refund-fixtures.jsonl ↓
    Build a test set for the explanation prompt from the fixtures' decision contexts. Generate cases for the open prompt (each input carries a decision, reason_code, and order_state), review them against the fixtures, then save them as the prompt's pinned test set.
    Fill in
    Number of Test Casespick a count covering approve / deny / manual_review reason codes, mirroring the decision contexts in refund-fixtures.jsonl
    Click Generate Test Cases
    You'll see: Generated cases appear, each pairing a decision context with the expected behavior that the explanation must restate the decision faithfully.
    Screen snapshot
    caliber · Library › Prompts › refund-explanation › Test Sets
    PromptsTest Sets
    Generate Test Cases
  16. Library › Prompts › refund-explanation › Test Sets creates · test set: refund-explanation faithfulness cases (pinned)
    Save the cases so the Runs stage can score the explanation against them. The expected behavior encodes faithfulness: the explanation must never upgrade, downgrade, or invent commitments beyond the fixed decision.
    Click Save to Test Sets
    You'll see: A "Saved to Test Sets" confirmation; the set is pinned to the refund-explanation prompt.
    Screen snapshot
    caliber · Library › Prompts › refund-explanation › Test Sets
    PromptsTest Sets
    Save to Test Sets
  17. Library › Prompts › refund-explanation › Runs creates · run: explanation faithfulness
    Run the explanation prompt against its pinned test set. The Runs stage executes the prompt and grades each case with the built-in pass/partial/fail judge, scoring whether the explanation stays faithful to the decision (the gate is ≥ 0.92). Run only after the deterministic refund lane above is green.
    Click Run tests
    You'll see: A scorecard with a high overall pass rate (target ≥ 0.92), confirming the explanation never contradicts the deterministic decision — completing the deterministic-first, explanation-last gate.
    Screen snapshot
    caliber · Library › Prompts › refund-explanation › Runs
    PromptsRuns
    Run tests

Assets (copy-paste)

The exact files this cookbook uses — copy each into the matching field. Source: docs-site/cookbooks/03-tool-hardening-contract-lab/assets/.

assets/tools/lookup-order.tool.json 
{
  "name": "lookup_order",
  "version": "1",
  "module_path": "caliber.workflows.demo_tools",
  "callable_name": "lookup_order",
  "side_effect_level": "read",
  "allow_in_preview": true,
  "input_schema": {
    "type": "object",
    "required": ["order_id"],
    "properties": {
      "order_id": {
        "type": "string",
        "description": "Order identifier to look up, e.g. \"ord_1120\"."
      }
    }
  },
  "output_schema": {
    "type": "object",
    "properties": {
      "order_id": {"type": "string"},
      "items": {"type": "array", "items": {"type": "string"}},
      "status": {"type": "string"}
    }
  },
  "notes": "Read-only lookup; runs live in the sandbox (allow_in_preview). The shipped callable returns {order_id, items, status} and takes ONLY order_id (scope fixture inputs to {\"order_id\": ...} or it raises a TypeError). Used here purely as the Tools-sandbox demo; it is NOT the source of decide_refund's inputs — the decide_refund python_code node parses order_state/risk_flags/amount/days_since_order from the JSON run input."
}
assets/tools/initiate-refund.tool.json 
{
  "name": "initiate_refund",
  "version": "1",
  "module_path": "caliber.workflows.demo_tools",
  "callable_name": "initiate_refund",
  "side_effect_level": "write",
  "input_schema": {
    "type": "object",
    "required": ["order_id"],
    "properties": {
      "order_id": {
        "type": "string",
        "description": "Order to refund."
      },
      "amount": {
        "type": "number",
        "description": "Refund amount in USD. Optional; the shipped callable ignores extras and returns its own amount_usd. Declared here so the workflow can pass the decided amount through the contract."
      }
    }
  },
  "output_schema": {
    "type": "object",
    "properties": {
      "order_id": {"type": "string"},
      "refund_status": {"type": "string"},
      "amount_usd": {"type": "number"}
    }
  },
  "notes": "side_effect_level=write -> in the Tools SANDBOX the response is mocked and the envelope carries a top-level mocked:true (it never really fires). In a normal WORKFLOW run there is no top-level mocked flag: the tool runs the real callable, but only AFTER the human_approval gate clears. Wire it AFTER the human_approval node so the refund only fires once the run is approved."
}
assets/dataset/refund-fixtures.jsonl 
{"id": "F01", "tags": ["golden", "approve"], "inputs": {"order_id": "ord_1120", "account_id": "acct_2101", "order_state": "delivered", "risk_flags": [], "amount": 49.0, "days_since_order": 5}, "expectations": {"decision": "approve", "reason_code": "ELIGIBLE_AUTO_APPROVE", "requires_approval": false}}
{"id": "F02", "tags": ["golden", "approve"], "inputs": {"order_id": "ord_1133", "account_id": "acct_2101", "order_state": "shipped", "risk_flags": [], "amount": 25.0, "days_since_order": 12}, "expectations": {"decision": "approve", "reason_code": "ELIGIBLE_AUTO_APPROVE", "requires_approval": false}}
{"id": "F03", "tags": ["golden", "deny"], "inputs": {"order_id": "ord_1002", "account_id": "acct_1432", "order_state": "cancelled", "risk_flags": [], "amount": 80.0, "days_since_order": 3}, "expectations": {"decision": "deny", "reason_code": "NOT_REFUNDABLE_STATE", "requires_approval": false}}
{"id": "F04", "tags": ["golden", "deny"], "inputs": {"order_id": "ord_0915", "account_id": "acct_1432", "order_state": "delivered", "risk_flags": [], "amount": 49.0, "days_since_order": 95}, "expectations": {"decision": "deny", "reason_code": "OUTSIDE_WINDOW", "requires_approval": false}}
{"id": "F05", "tags": ["edge", "boundary", "approve"], "inputs": {"order_id": "ord_1140", "account_id": "acct_3300", "order_state": "delivered", "risk_flags": [], "amount": 200.0, "days_since_order": 30}, "expectations": {"decision": "approve", "reason_code": "ELIGIBLE_AUTO_APPROVE", "requires_approval": false}}
{"id": "F06", "tags": ["edge", "boundary", "manual_review"], "inputs": {"order_id": "ord_1141", "account_id": "acct_3300", "order_state": "delivered", "risk_flags": [], "amount": 250.0, "days_since_order": 10}, "expectations": {"decision": "manual_review", "reason_code": "AMOUNT_OVER_THRESHOLD", "requires_approval": true}}
{"id": "F07", "tags": ["edge", "boundary", "deny"], "inputs": {"order_id": "ord_1150", "account_id": "acct_3300", "order_state": "delivered", "risk_flags": [], "amount": 49.0, "days_since_order": 31}, "expectations": {"decision": "deny", "reason_code": "OUTSIDE_WINDOW", "requires_approval": false}}
{"id": "F08", "tags": ["negative", "fraud", "manual_review"], "inputs": {"order_id": "ord_1166", "account_id": "acct_9001", "order_state": "delivered", "risk_flags": ["fraud_suspected"], "amount": 49.0, "days_since_order": 4}, "expectations": {"decision": "manual_review", "reason_code": "RISK_FLAG_PRESENT", "requires_approval": true}}
{"id": "F09", "tags": ["negative", "missing_data", "fail_closed", "manual_review"], "inputs": {"order_id": "ord_1177", "account_id": "acct_9001", "order_state": "delivered", "risk_flags": null, "amount": 49.0, "days_since_order": 4}, "expectations": {"decision": "manual_review", "reason_code": "MISSING_RISK_DATA", "requires_approval": true}}
assets/tools/decide_refund.py 
"""Deterministic refund-eligibility decision for a workflow ``python_code`` node.

This is NOT a registered tool. Paste the body of this module into a CALIBER
**Python Code** node (``Compose → Workflows`` → drag a *Python Code* node). It
needs no registration, versions with the workflow, and uses **stdlib only**.
Wire it as the FIRST node after START: ``START -> decide_refund``. A python_code
node receives the run input as an UNPARSED string in ``run_input``, so this
entrypoint parses it as JSON (when it looks like a JSON object) and reads the
decision fields from that dict. (``json`` is pre-injected into the node sandbox
-- do NOT add a module-level ``import json``.)

Run-input fields (parsed from the run input JSON object):
    order_state       str   -- order lifecycle, e.g. "delivered" | "shipped" |
                               "cancelled" | "returned" | "" (unknown).
    risk_flags        list  -- fraud/risk signals, e.g. ["fraud_suspected"],
                               ["chargeback_history"], or [] (none). May be
                               ``None`` when the risk lookup failed -> we
                               FAIL CLOSED to manual_review (see RULES below).
    amount            float -- refund amount in USD (>= 0).
    days_since_order  int   -- whole days between the order date and now.

Node outputs:
    decision          str   -- one of: "approve" | "deny" | "manual_review".
    reason_code       str   -- stable machine code (see REASON_CODES); pairs
                               with the policy-reason-normalizer skill.
    requires_approval bool  -- True when a human_approval gate must clear before
                               initiate_refund runs. Always True for
                               manual_review; never True for a clean approve.

Deterministic rules (evaluated top-to-bottom; first match wins):
    1. FAIL CLOSED: risk data missing (risk_flags is None) OR order_state is
       empty/unknown -> manual_review / requires_approval=True.
    2. Any fraud/risk flag present -> manual_review / requires_approval=True.
    3. Order not in a refundable state (only "delivered"/"shipped"/"returned"
       are refundable) -> deny.
    4. Outside the refund window (days_since_order > REFUND_WINDOW_DAYS) -> deny.
    5. Negative/zero amount -> deny (nothing to refund).
    6. Amount over the auto-approve threshold (amount > AUTO_APPROVE_MAX_USD)
       -> manual_review / requires_approval=True.
    7. Otherwise (in window, no risk, small amount) -> approve /
       requires_approval=False.

The rule_checks in verification.yaml
(``deterministic_decision_preserved`` / ``approval_required_for_high_risk``)
are enforced HERE plus by the downstream human_approval gate -- not by a judge.
"""

# --- Policy constants (the only knobs; keep them visible at the top) ---------
REFUND_WINDOW_DAYS = 30          # matches demo_tools.lookup_policy
AUTO_APPROVE_MAX_USD = 200.0     # over this -> human approval required
REFUNDABLE_STATES = {"delivered", "shipped", "returned"}

# Stable reason codes (machine-facing; the normalizer skill maps these to copy)
REASON_CODES = {
    "MISSING_RISK_DATA": "Risk signals unavailable; failing closed to review.",
    "UNKNOWN_ORDER_STATE": "Order state unknown; failing closed to review.",
    "RISK_FLAG_PRESENT": "Risk/fraud flag on the account; routing to review.",
    "NOT_REFUNDABLE_STATE": "Order is not in a refundable state.",
    "OUTSIDE_WINDOW": "Past the refund window.",
    "INVALID_AMOUNT": "Refund amount is zero or negative.",
    "AMOUNT_OVER_THRESHOLD": "Amount exceeds the auto-approve limit; needs review.",
    "ELIGIBLE_AUTO_APPROVE": "In window, no risk, within auto-approve limit.",
}


def decide_refund(
    order_state: str,
    risk_flags,
    amount,
    days_since_order,
) -> dict:
    """Pure, deterministic refund decision. Returns the node-output dict."""

    def out(decision: str, reason_code: str, requires_approval: bool) -> dict:
        return {
            "decision": decision,
            "reason_code": reason_code,
            "requires_approval": bool(requires_approval),
        }

    # Rule 1 -- fail closed when inputs are missing/unusable.
    if risk_flags is None:
        return out("manual_review", "MISSING_RISK_DATA", True)
    state = (order_state or "").strip().lower()
    if not state:
        return out("manual_review", "UNKNOWN_ORDER_STATE", True)

    # Coerce numerics defensively (still fail closed on garbage).
    try:
        amount_val = float(amount)
        days = int(days_since_order)
    except (TypeError, ValueError):
        return out("manual_review", "MISSING_RISK_DATA", True)

    # Rule 2 -- any risk flag -> review + approval.
    if any(str(flag).strip() for flag in risk_flags):
        return out("manual_review", "RISK_FLAG_PRESENT", True)

    # Rule 3 -- non-refundable lifecycle state.
    if state not in REFUNDABLE_STATES:
        return out("deny", "NOT_REFUNDABLE_STATE", False)

    # Rule 4 -- outside the refund window.
    if days > REFUND_WINDOW_DAYS:
        return out("deny", "OUTSIDE_WINDOW", False)

    # Rule 5 -- nothing to refund.
    if amount_val <= 0:
        return out("deny", "INVALID_AMOUNT", False)

    # Rule 6 -- large refund needs a human.
    if amount_val > AUTO_APPROVE_MAX_USD:
        return out("manual_review", "AMOUNT_OVER_THRESHOLD", True)

    # Rule 7 -- clean auto-approve.
    return out("approve", "ELIGIBLE_AUTO_APPROVE", False)


# --- python_code node entrypoint --------------------------------------------
# A CALIBER Python Code node calls ``run_python_node(...)`` and uses its RETURN
# value as the node output. A module-level ``result = ...`` is DISCARDED (the
# runtime wraps a body lacking this def in a function with no return), so define
# the entrypoint explicitly and return both output ports.
#
# ``json`` is PRE-INJECTED into the node sandbox; do NOT ``import json`` here.
# Because decide_refund is the first node after START, its inputs arrive as the
# run input -- an UNPARSED string in ``run_input``. Parse it as JSON when it
# looks like a JSON object, then read the decision fields from that dict.


def run_python_node(input=None, context=None, inputs=None, run_input=""):
    if isinstance(run_input, str) and run_input.strip().startswith("{"):
        data = json.loads(run_input)
    else:
        data = run_input or {}
    if not isinstance(data, dict):
        data = {}
    result = decide_refund(
        order_state=data.get("order_state", ""),
        risk_flags=data.get("risk_flags"),  # absent -> None -> fail closed
        amount=data.get("amount", 0),
        days_since_order=data.get("days_since_order", 0),
    )
    return {"text": json.dumps(result), "result": result}
assets/prompts/refund-explanation.md 
---
name: refund-explanation
model_hint: a small/cheap instruct model is fine (explanation only, no decision)
variables: [decision, reason_code, order_state, customer_name]
commit_message: "v1 explanation layer — explains, never overrides, the deterministic decision"
---

You write a short, customer-facing explanation of a refund decision that has
ALREADY been made by a deterministic policy engine. You do NOT make or change
the decision. Your only job is to put the given decision into plain, respectful
language.

The decision is fixed and authoritative:
- decision: {{ decision }}            (one of: approve, deny, manual_review)
- reason_code: {{ reason_code }}      (the machine reason for that decision)
- order_state: {{ order_state }}
- customer_name: {{ customer_name }}

Hard rules:
- NEVER contradict, soften, or upgrade/downgrade {{ decision }}. If it is
  "deny", do not imply the refund might still happen. If it is "manual_review",
  say it is under review and a person will follow up — do NOT promise an outcome.
  If it is "approve", confirm the refund is being processed.
- Make NO new commitments: no amounts, no dates, no timelines, no exceptions,
  no policy promises that are not implied by {{ decision }} and {{ reason_code }}.
- Do not invent facts about the order beyond {{ order_state }} and the reason.
- Address {{ customer_name }} once, warmly and briefly. 2–4 sentences. Plain
  text, no JSON, no markdown.

Write the explanation now.
assets/skills/policy-reason-normalizer.md 
---
name: policy-reason-normalizer
summary: Map a machine refund reason_code to one stable, human-readable reason phrase — without changing the decision.
---

# Policy Reason Normalizer

Use this skill whenever a refund decision carries a machine `reason_code` that
needs to be stated to a person (customer message, review note, audit log). It
normalizes the code to ONE canonical sentence. It never alters the `decision`
and never adds commitments.

## Canonical mapping

| reason_code | Canonical reason phrase |
| --- | --- |
| `ELIGIBLE_AUTO_APPROVE` | The order is within the refund window with no risk flags, so the refund was approved automatically. |
| `AMOUNT_OVER_THRESHOLD` | The refund amount is above the automatic-approval limit, so it was sent for human review. |
| `RISK_FLAG_PRESENT` | A risk or fraud signal is associated with the account, so the refund was sent for human review. |
| `MISSING_RISK_DATA` | Required risk information was unavailable, so the refund was sent for human review to stay safe. |
| `UNKNOWN_ORDER_STATE` | The order's status could not be confirmed, so the refund was sent for human review. |
| `NOT_REFUNDABLE_STATE` | The order is not in a state that can be refunded, so the refund was declined. |
| `OUTSIDE_WINDOW` | The order is past the refund window, so the refund was declined. |
| `INVALID_AMOUNT` | There is no positive amount to refund, so the refund was declined. |

## Rules

- Output exactly one phrase from the table for the given `reason_code`.
- If the `reason_code` is unrecognized, output: "This decision was made by
  policy; please consult the review queue for details." — do not guess.
- Do not state or imply an outcome that conflicts with the `decision`
  (`approve` / `deny` / `manual_review`).
- Add no amounts, dates, or timelines. The phrase is a reason, not a promise.
assets/skills/customer-safe-refund-language.md 
---
name: customer-safe-refund-language
summary: Guardrails for phrasing a refund decision to a customer safely — respectful tone, no new promises, never override the decision.
---

# Customer-Safe Refund Language

Apply this skill when turning a refund `decision` into customer-facing wording.
It governs TONE and SAFETY only; the `decision` and `reason_code` are inputs
you must preserve exactly.

## Allowed phrasing by decision

- **approve** — Confirm warmly that the refund is being processed. You may say
  it has been approved. Do not invent the exact arrival date or amount unless
  it was explicitly provided.
- **deny** — State plainly and kindly that the refund cannot be processed, and
  give the canonical reason. Do not imply it might still happen, and do not
  invite an appeal that policy does not offer.
- **manual_review** — Say the request is being reviewed and a team member will
  follow up. Do NOT predict the outcome (no "you'll likely get it"), no ETA.

## Never do

- Never change, hedge, or reverse the `decision`.
- Never promise amounts, dates, timelines, credits, or exceptions that are not
  already implied by `decision` + `reason_code`.
- Never blame the customer or disclose internal risk/fraud signals; for
  risk-driven reviews say only that "additional review" is required.
- Never output JSON or internal codes to the customer — use plain language.

## Style

Address the customer by name once, keep it to 2–4 sentences, be courteous and
concrete. When in doubt, say less rather than promise more.

Evaluation & quality gates

Quality gateTarget
Deterministic fixture pass rate≥ 0.97; decision mismatch = 0
Approval enforced on high-riskvisible in the run timeline
Explanation faithfulness≥ 0.92
Developer notes & gotchas
  • A registered tool needs an importable module_path+callable_name; the shipped demo_tools callables let you do this from the UI with no code.
  • Field is side_effect_level (read/write/external_action); read tools need allow_in_preview: true to run live in the sandbox.
  • The decision table is structured workflow source: versioned, inspectable, and executable without custom Python.

CALIBER : Contextual Adaptive Lifecycle for Intelligent Build, Evaluation, and Refinement — Cookbooks — every recipe is UI-implementable on the shipped platform. Source + assets under docs-site/cookbooks/<nn>-…/. Regenerate with python3 docs-site/cookbooks/training/build.py.