CALIBER Platform Architecture

The caliber module is the system-level control plane for the product. Its create_app() factory is used in two ways: MLflow can load it as an in-process mlflow.app, or Uvicorn can serve it independently while it uses MLflow's HTTP APIs. The first topology shares MLflow's process failure domain; the second does not. Its responsibilities are correspondingly broad — wider than those of any single feature module — and span the following concerns:

• It bootstraps the Starlette application and the shared runtime dependencies on which every feature depends.
• It serves the React SPA under /caliber/.
• It exposes the CALIBER API surface under /ajax-api/2.0/mlflow/caliber/*.
• It provides shared persistence, authorization, CSRF, rate-limiting, project- visibility, and audit primitives that feature paths wire explicitly.
• It orchestrates up to nine in-process loops for refinement, tool calibration, workflow runs, Aria plans, knowledge-base builds, workflow scheduling, janitor work, release reconciliation, and webhook dispatch/recovery.
• It integrates with MLflow tracing, the MLflow prompt registry, object storage, and optional event-bus backends.

These responsibilities are realized across a small set of primary code paths, which serve as the entry points for the rest of this document:

• caliber/src/caliber/server.py
• caliber/src/caliber/routes/__init__.py
• caliber/src/caliber/config.py
• caliber/src/caliber/db/session.py
• caliber/src/caliber/auth.py
• caliber/src/caliber/routes/static.py
• caliber/caliber-ui/src/components/AppShell.tsx

Given that breadth, CALIBER is best understood as a layered application rather than a monolith stitched together from feature-specific entry points. Each layer has a clear responsibility and consumes the layers beneath it through stable interfaces.

The architectural boundary that matters most is that feature modules do not own top-level application bootstrapping. server.py constructs core app-lifetime dependencies and stores them on app.state; some feature services, runtime services, and backends are instead constructed lazily or per operation. Core lifecycle ownership remains in create_app() without implying that every route dependency is a process-lifetime singleton.

The layers described above resolve, at runtime, into the flow shown below: a browser drives the SPA and static surfaces, requests fan into the route modules, and the route modules and background workers share access to persistence, MLflow, object storage, the event bus, and the provider adapters.

flowchart LR
    B[Browser]:::user
    SPA[React SPA<br/>/caliber/]:::ui
    ST[Static route handler]:::ctrl
    API[Starlette route modules]:::ctrl
    AUTH[Auth, CSRF, rate limit, project scoping]:::ctrl
    DB[(SQLAlchemy metadata DB)]:::store
    ML[MLflow APIs and traces]:::ext
    OBJ[(Object store / S3 / MinIO)]:::store
    BUS[Event bus]:::async
    WK[Background workers]:::async
    EXT[Provider adapters<br/>LLM, eval, promoter, MCP, sandbox]:::ext

    B --> SPA
    B --> ST
    SPA --> API
    ST --> API
    API --> AUTH
    AUTH --> DB
    API --> DB
    API --> ML
    API --> OBJ
    API --> EXT
    API --> BUS
    WK --> DB
    WK --> ML
    WK --> OBJ
    WK --> BUS
    WK --> EXT

Several structural properties follow from these topologies and are worth making explicit, because they shape every design decision downstream:

• Embedded mode loads CALIBER into the MLflow server as a sibling ASGI surface. Standalone mode serves the same application separately and points MLFLOW_TRACKING_URI at MLflow. Neither mode makes CALIBER a transparent gateway in front of MLflow.
• The frontend shell is bundled separately with Vite, but it is served through the CALIBER package by routes/static.py rather than from a distinct origin.
• Core app-lifetime dependencies are constructed in create_app() and exposed through app.state, including the SQLAlchemy engine and session factory, configured storage and provider adapters, the event bus, and worker lifecycle. Some feature/runtime services and backends are constructed lazily or per operation instead of being retained as app-lifetime singletons.
• CALIBER_DATABASE_URL independently owns CALIBER's tables. It is not required to equal MLflow's backend-store URL; the bundled stack uses separate logical databases so the two Alembic histories never compete for one version table.
• The codebase uses sync SQLAlchemy consistently even though route callables are predominantly async. Selected blocking work is explicitly sent through run_in_threadpool or asyncio.to_thread; there is no async ORM. An async route is not automatically moved to Starlette's threadpool merely because it uses a synchronous session.

With the runtime established, the next question is where authoritative state lives. The persistence model is deliberately broad: CALIBER keeps durable product state in SQLAlchemy rows and treats external systems such as MLflow and object storage as integrated subsystems rather than as replacements for relational metadata.

That state divides into the following domains, each anchored to a representative set of tables:

The division of authority across these domains is governed by a small set of ownership rules, and they hold consistently across the platform:

• Relational metadata is the authoritative control plane.
• Object storage is authoritative for file bytes, but not for the file inventory.
• MLflow is authoritative for prompt registry versions and traces, but not for CALIBER-specific workflow, tool, or skill metadata.
• Route handlers and workers both mutate the same tables. Durable queue and run arbitration uses explicit status transitions, conditional claims, leases, and timeline rows. Live SSE fanout and lifecycle stop state may remain process-local or use a configured NATS, Redis, or database event transport; not every coordination mechanism is durable, and the workers normally share the ASGI process.

Versioning is a cross-artifact UI concern, not one shared persistence or release contract. The normalized frontend model maps several deliberately different idioms. In the final column, gate means a release/deployment or persisted version-panel gate. That is distinct from the enforced evaluation gate inside the prompt and skill refinement workers, whose passing outcome is candidate_ready, and from the prompt panel's advisory verdict, which does not block alias rotation:

The shared VersionPanel is mounted for prompts, workflows, knowledge bases, skills, and tools through per-artifact adapters; sharing the component does not make their guarantees uniform. The read-only /releases/timeline aggregates audited release actions, while /releases/live enumerates the database-backed workflow deployments and active knowledge-base versions. Prompt liveness remains in MLflow and is shown on the per-prompt page rather than inferred by that aggregate.

GET /capabilities publishes the same distinction as a machine-readable artifact_families contract for all nine families: history idiom, live-target shape, promotion/rollback/evidence flags, gate mode, and calibration idiom. Contract tests require every family to declare every field. This prevents clients from inferring semantics from the shared panel, but it is not yet a proof that every family route implements a common Releasable or Rollbackable interface.

Human and API-client interactions enter through the HTTP surface. That surface is centralized in routes/__init__.py, which registers each feature module under the same-origin AJAX namespace; in-process workers consume the underlying services and stores directly. The endpoints below are representative rather than exhaustive:

• /prompts
• /tools
• /skills
• /mcp-servers
• /agents
• /workflows
• /workflow-versions/*
• /workflow-runs/*
• /assistant/*
• /aria/plans/* (Aria goal-plan orchestration)
• /judges, /review-queues, /eval-datasets, /gate-verdicts/*
• /releases/timeline, /releases/live (cross-artifact releases & rollback)
• /releases/operations, /releases/operations/reconcile (operator-visible prompt release recovery)

On the client side, the corresponding entry points are organized as page modules, each of which owns a single feature surface:

• caliber/caliber-ui/src/pages/Prompts.tsx
• caliber/caliber-ui/src/pages/ToolRegistry.tsx
• caliber/caliber-ui/src/pages/Skills.tsx
• caliber/caliber-ui/src/pages/McpServers.tsx
• caliber/caliber-ui/src/pages/Agents.tsx
• caliber/caliber-ui/src/pages/AgentDetail.tsx
• caliber/caliber-ui/src/pages/Workflows.tsx
• caliber/caliber-ui/src/pages/WorkflowEditor.tsx
• caliber/caliber-ui/src/pages/Settings.tsx

The frontend shell never talks to databases or providers directly. Every user action routes through the CALIBER HTTP surface, even when the eventual effect is a provider call, an object-store mutation, or a workflow enqueue. The shared API boundary provides common authorization, visibility, and audit primitives, but each feature path must wire them explicitly; their presence is not a repository- wide uniform-coverage guarantee.

The top bar exposes the active workspace rather than leaving project scoping as an API-only header. An operator can create and immediately select a project; the API client then sends its id as X-CALIBER-Project, invalidates scoped queries, and uses that project for newly created workflows and managed files.

The API surface admits two broad shapes of work: bounded validation, external calls, and many durable mutations that complete on the request path; and explicitly queued or long-running work handled by an in-process loop. The sequence below shows both branches and the point at which they diverge.

All configured loops share background_tasks_enabled as their lifecycle gate. WorkflowRunWorker, KnowledgeBaseWorker, and WorkflowScheduler additionally have independent enable flags; the other five loops, including AriaPlanWorker, do not.

The loops are in-process, so each server worker process runs its own full set. mlflow server defaults to four gunicorn workers, which means all nine loops exist four times over. Where arbitration is durable that is safe: queue consumers so exactly one process wins each row, and the cron scheduler is idempotent by a minute-bucketed key backed by a unique partial index. Two consequences do not follow that rule and are worth sizing for:

• Worker identity must survive forking. A heartbeat row in caliber_worker_heartbeats is keyed by a worker id built from the process id as well as the instance (worker_registry.new_worker_id), because that key is shared by every process pointed at the same database. It is deliberately not a fresh UUID per start: an unclean exit leaves its row behind as the outage signal, so a random id would accumulate one stale row per crash-restart.
• In-memory limits are per process. The failed-login throttle (routes/auth.py) and the API RateLimiter (rate_limit.py) are token/attempt buckets behind a thread lock, not shared state, so their effective ceiling scales with the worker count. Size the configured limits accordingly, or pin the server to one worker (MLFLOW_WORKERS=1 for scripts/run-dev.sh, --workers 1 otherwise). See the layered architecture for the same point at the platform tier.

sequenceDiagram
    participant U as User
    participant UI as React SPA
    participant RT as Route handler
    participant DB as SQLAlchemy DB
    participant BUS as Event bus
    participant WK as Background worker
    participant EXT as External subsystem

    U->>UI: Interact with CALIBER page
    UI->>RT: Same-origin API request
    RT->>RT: Auth, scope, CSRF, validation
    RT->>DB: Persist or read control-plane state
    alt Request-path action only
        RT->>EXT: Call MLflow, sandbox, storage, or gateway
        RT-->>UI: Return response
    else Queued background action
        RT->>BUS: Publish event if configured
        RT-->>UI: Return accepted / queued response
        WK->>DB: Claim queued work
        WK->>EXT: Execute provider/runtime work
        WK->>DB: Persist final state, events, and summaries
    end

The same lifecycle plays out at three scales — process startup, individual request handling, and shutdown — and each has well-defined checkpoints:

• At application startup, configuration is loaded and validated, logging and tracing are configured, core engine/session, provider, storage, and event-bus dependencies are built, and enabled workers are started by the Starlette lifespan manager. Other operation-scoped services remain lazy.
• During request handling, the route module validates the headers, body, and query, uses app-lifetime or operation-scoped dependencies together with a DB session, and may mutate state inline, explicitly offload blocking work, or enqueue durable work.
• At shutdown, workers receive a stop signal and bounded grace before the event bus and SQLAlchemy engine are torn down. Most loops settle or release their durable claims. Tool calibration immediately fences its active generation, waits at most its grace for the tracked drain, and performs no stop-time database settlement. An interrupted claim may remain visibly running/ambiguous, while the retained fence rejects any late terminal persistence. This is bounded shutdown rather than an unconditional drain guarantee.

CALIBER implements its own default session login in both deployment topologies. Credentials are checked against scrypt hashes in the account table, and successful login creates a revocable server-side session carried by an HttpOnly cookie. The product default CALIBER_AUTH_BOOTSTRAP_ALLOW_INSECURE_DEFAULT=false does not create a known credential. The native launcher opts into admin / admin only when MLflow binds to loopback, and the bundled Compose stack opts in only while every published port is pinned to 127.0.0.1; the operator must replace that credential immediately. The bootstrap runs only while the account table is empty and never resets an existing account. Any network-reachable deployment must keep the opt-in false, configure a strong bootstrap password source, and use TLS/Secure cookies. Normal account creation and reset continue to reject admin and other weak passwords.

An installation that already has an identity-aware proxy can explicitly select trusted_header mode instead. In that mode, and only that mode, request headers provide the identity:

• The X-CALIBER-User header identifies the caller.
• The X-CALIBER-Project header supplies the active project for multi-project scoping.
• auth.py resolves the viewer, operator, approver, and admin scopes.
• Route handlers call require_user() or require_scopes() before any mutation.

The shipped local launchers assign admin every scope. Before exposing a deployment, set CALIBER_BOOTSTRAP_PASSWORD to a strong value before first boot or reset the password in Administration; password changes revoke the account's existing sessions.

Layered over this identity model is a set of controls that protect both the request path and the data it touches:

• CSRF protection guards browser-driven state changes.
• Optional rate-limiting middleware bounds request volume.
• Project and visibility filters are applied on the feature paths that wire the shared scoping helpers.
• Implemented mutation paths append audit records; database-local mutation and audit can share one caller transaction. Prompt alias changes additionally commit an exact release intent before the MLflow effect and expose incomplete operations for reconciliation. Other external effects and separately committed paths must be evaluated individually and may remain unrecorded dual-write boundaries.
• Secrets are referenced indirectly through *_source fields rather than carried as raw secret material.
• Manifest validation rejects inline secrets in workflow definitions.
• User-authored Python Code, Aria draft tests, and registered-tool execution use a short-lived local subprocess with an empty environment, private working directory, bounded output, process-group termination, and best-effort POSIX resource limits. The child signals ready and waits for the parent to start authored work; the parent then owns the exact runtime deadline and kills the process group, while an uncatchable child watchdog independently covers source/module resolution, inspection, every test case, invocation, result representation, and serialization. A positive startup grace is a separate pre-ready allowance; with zero grace, startup consumes the caller's overall budget. Sandbox HTTP execution runs off the ASGI event loop. This is process containment, not a container, VM, or kernel sandbox.

The dominant trust boundary runs between control-plane state and code execution. Route handlers may accept user-authored manifests, prompt text, tool metadata, and skill content, but only specific runtime paths are permitted to execute code or make external calls. The local subprocess contains Python Code, Aria draft, and default registered-tool execution; it still retains ambient host filesystem and network authority. external_app callables remain trusted-operator code and need a separate worker/container boundary before mutually untrusted authors can use them safely. MCP admission is separately mediated by command/host allowlists and deployment preflight; local stdio containment is likewise not an OS sandbox. These controls are path-specific: protected routes select require_user() or require_scopes() according to their operation, while public health and published-service paths use separate admission rules. Importing a shared guard, scoping helper, or audit function does not prove complete coverage for a module.

Operating CALIBER depends on an observability spine that is shared across every module rather than reimplemented per feature. Its core pieces are:

• structured JSON logging in observability/logging.py;
• guarded MLflow tracing in observability/mlflow_tracing.py;
• request trace-ID propagation in observability/trace.py;
• Prometheus metrics exposed on the /metrics route;
• workflow-run events and checkpoints, which serve as durable operational timeline state.

The operational design choices behind these pieces favor graceful degradation, so that observability and serving infrastructure fail soft rather than taking the request path down with them:

• Logging defaults to stderr, with optional mirroring to S3 or the Object Store.
• Tracing is on-but-inert by default and degrades to a no-op when MLflow tracing support is unavailable.
• Event publication is best-effort and must never break the request path.
• Static UI serving degrades gracefully, returning a 503 with an operator-facing message when the bundle is absent.

CALIBER is built to be extended along its existing seams rather than rewired. The primary extension points follow the layering described earlier:

• A new route module is added and registered in routes/__init__.py.
• A new provider adapter is added and wired through server.py.
• New worker loops are added into the lifespan graph.
• New storage or event-bus backends are added behind the existing abstractions.
• New pages extend the React SPA under caliber-ui/src/pages.

Set against those seams are the architectural constraints the system currently accepts, stated plainly so that future work can weigh them deliberately:

• Embedded mode couples CALIBER and MLflow to one process failure domain; standalone mode trades that coupling for an HTTP dependency and a second service process.
• SQLAlchemy usage is sync throughout the server.
• Many feature modules still perform orchestration directly in the route files rather than through deeper service layers.
• Several domains intentionally use hidden runtime identities (prompt_targets, skill_targets) so that they fit into the shared refinement machinery.
• Workflow execution and assistant logic remain substantially in-process, even though both are structured well enough to be extracted later.

Taken together, the picture is consistent: CALIBER is one ASGI control-plane codebase, deployable embedded or standalone, with feature modules layered on a common runtime substrate. The per-feature documents take it from here, describing how each major bounded context plugs into that substrate.