MCP Architecture

The MCP module integrates external Model Context Protocol (MCP) servers into CALIBER. It treats those servers as operator-managed runtime dependencies whose tools can be discovered, policy-controlled, tested, calibrated, and invoked from both the user interface and the workflow runtime. The module's purpose is to make a remote, externally owned tool surface behave like a governed, inspectable part of the CALIBER control plane.

From that purpose follow the module's responsibilities. It registers and maintains MCP server connection metadata; it speaks the MCP transports through a shared gateway layer; it discovers and caches the remote tool inventory of each server; it maintains the per-tool policy, the saved test cases, and the calibration results that CALIBER layers on top of those tools; it exposes one-off invocation and test-connection APIs for operators; and it provides synchronous invocation helpers that the workflow runtime binds against.

This work is carried by the following primary code paths, spanning the route layer, the transport gateway, the registry model, the bundled first-party server, and the front end:

• caliber/src/caliber/routes/mcp_servers.py
• caliber/src/caliber/mcp_gateway.py
• caliber/src/caliber/mcp_policy.py
• caliber/src/caliber/db/models.py (CaliberMcpServer)
• caliber/src/caliber/mcp_servers/db/*
• caliber/caliber-ui/src/pages/McpServers.tsx

The module keeps persistence, transport, and runtime invocation in separate hands so that none of them has to reimplement the others. The table below records the owners:

This split is explicit and intentional. The routes own persistence, the policy module owns admission and deployment preflight, the gateway owns network and transport mechanics; and the workflow runtime reaches MCP tools only through sync wrappers, so that the same tools are callable from the synchronous interpreter path without that path ever touching transport details directly.

Discovery is inventory, not authorization. A newly discovered tool is denied until an administrator saves an explicit allowed, side_effect_level, and requires_approval classification. The same admission check protects Playground invocation, calibration, workflow runtime, and deployment; bundled database presets arrive with explicit policies rather than relying on a name heuristic.

The diagram below shows how those owners connect. Operator actions flow from the UI through the routes to the registry and the gateway, while the workflow runtime reaches the same gateway directly. The gateway, in turn, drives the official SDK out to the external servers.

flowchart LR
    UI[MCP Servers UI]:::ui
    API[routes/mcp_servers.py]:::ctrl
    DB[(CaliberMcpServer)]:::store
    POL[mcp_policy.py]:::ctrl
    GW[mcp_gateway.py]:::ctrl
    SDK[Official MCP Python SDK]:::ext
    EXT[External MCP servers]:::ext
    WF[Workflow runtime]:::ctrl

    UI --> API
    API --> DB
    API --> POL
    POL --> GW
    GW --> SDK
    SDK --> EXT
    WF --> POL
    POL --> GW
    GW --> DB

A few structural properties make this layout work. The same policy and gateway code paths serves both operator tooling and workflow runtime invocation, so transport behavior cannot drift between the two. Discovered tool metadata is cached on the CaliberMcpServer row rather than fetched on every UI render, which keeps the interface responsive and avoids hammering remote servers. Policies are CALIBER-local overlays that sit on top of the remotely discovered tools rather than properties of the remote server itself. And the gateway can either bootstrap a server's configuration from its DB row or accept an already-loaded configuration object, so both stored servers and transient ones run through one implementation.

All of this state lives on a single carrier, the CaliberMcpServer row, whose fields divide cleanly by purpose:

This division reflects a clear ownership split. Remote servers remain authoritative for the live tool implementation, so CALIBER never pretends to define what a tool does. CALIBER, in turn, becomes authoritative for the local policy, the test fixtures, and the calibration summaries it maintains over those tools. Because discovery is explicit and cached rather than continuous, the UI stays fast and the system avoids paying repeated transport overhead on every page load.

The module's HTTP surface mirrors the lifecycle of a managed server. All routes are mounted under /ajax-api/2.0/mlflow/caliber and are shown relative to that prefix throughout.

Registry management covers the create, read, update, and delete operations over server records:

• GET /mcp-servers
• POST /mcp-servers
• GET /mcp-servers/{server_id}
• PATCH /mcp-servers/{server_id}
• DELETE /mcp-servers/{server_id} — guarded: returns 409 when the server is still referenced by an active deployment, and snapshots the full server definition into the delete audit row so it can be recreated.

Transport validation and discovery exercise the real connection: testing it, refreshing the cached tool inventory, and reading that inventory back:

• POST /mcp-servers/{server_id}/test-connection
• POST /mcp-servers/{server_id}/discover-tools
• GET /mcp-servers/{server_id}/tools

Per-tool policy and validation manage the CALIBER-local overlay for an individual tool, including its policy, its saved test cases, and its calibration:

• PATCH /mcp-servers/{server_id}/tools/{tool_name}/policy
• PUT /mcp-servers/{server_id}/tools/{tool_name}/test-cases
• POST /mcp-servers/{server_id}/tools/{tool_name}/calibrate

Playground invocation supports a single ad hoc tool call for inspection and experimentation:

• POST /mcp-servers/{server_id}/invoke-tool

The front-end entry point that drives all of the above is:

• caliber/caliber-ui/src/pages/McpServers.tsx

The endpoints above combine into a typical operator session: register a server, validate it and discover its tools, then invoke or calibrate those tools. The sequence below traces that path through the routes, the registry, the gateway, and the external server.

sequenceDiagram
    participant U as Operator
    participant UI as MCP UI
    participant API as routes/mcp_servers.py
    participant DB as CALIBER DB
    participant GW as mcp_gateway.py
    participant MCP as External MCP server

    U->>UI: Register server config
    UI->>API: POST /mcp-servers
    API->>DB: Insert CaliberMcpServer row
    API-->>UI: Return registry row

    U->>UI: Test connection or discover tools
    UI->>API: POST /test-connection or /discover-tools
    API->>GW: Build gateway config and connect
    GW->>MCP: tools/list
    MCP-->>GW: Tool catalog
    GW-->>API: Normalized tools
    API->>DB: Persist discovered_tools, status, last_connected_at
    API-->>UI: Return discovery result

    U->>UI: Invoke or calibrate tool
    UI->>API: POST /invoke-tool or /calibrate
    API->>DB: Load effective policy and saved cases
    API->>GW: call_tool through central admission
    GW->>MCP: tools/call
    API->>DB: Persist calibration aggregate if applicable

A handful of runtime rules govern that flow. A connection test validates the minimal configuration first and only then performs a real tools/list round trip, so misconfiguration is caught before any network cost is paid. Invocation and calibration both run through the same _invoke_mcp_tool() path, which keeps the known-tool checks, the policy checks, the timeouts, and the gateway behavior consistent regardless of which entry point triggered the call. The gateway rechecks status, discovered-tool membership, allow/deny policy, transport allowlists, and rate limits, so workflow runtime calls cannot bypass the route layer. The synchronous interpreter therefore never duplicates either policy or transport logic.

Because every operation in this module can reach out to an externally owned server, access control is weighted toward the operations that cross that boundary. The following controls apply:

• Server registration and mutation — create, update, and delete — require the admin scope.
• Connection tests, tool discovery, tool-policy updates, and direct tool invocation also require the admin scope; because each of these reaches out to the remote server, they are deliberately treated as the highest-privilege operations.
• Only the saved test cases and calibration endpoints are operator-scoped.
• Policy overlays can explicitly block a remote tool even when the remote server still advertises it.
• Invocation is fail closed when the server is not active, discovery has not established the tool inventory, the requested tool is outside that inventory, or its policy blocks it. Per-tool minute limits are enforced in process; they are not a distributed quota across replicas.
• Stdio execution accepts only explicitly configured executables. The default permits ${PYTHON} only for allowlisted -m modules, rejects Python -c and unlisted scripts, blocks process-injection environment variables, avoids a shell, replaces every environment key the MCP SDK would otherwise inherit, uses an operator-controlled safe PATH, resolves launchers to absolute paths, and starts in a fresh private working directory.
• Remote transports require an exact host allowlist. Non-loopback plain HTTP is rejected unless an operator explicitly enables it; embedded URI credentials are rejected.
• The UI supports no authentication or an environment-backed bearer token. Basic and custom-header auth remain API-configured for existing integrations; OAuth is rejected because CALIBER does not implement a token exchange/refresh flow. Terminate OAuth in an operator-managed proxy instead.
• Workflow promotion and gated-promotion approval re-run MCP dependency preflight. Missing/inactive servers, stale or absent live discovery, unknown or blocked tools, tools without explicit allowed, side_effect_level, and requires_approval classifications, under-classified side effects, and missing approval bindings prevent alias rotation. Catalog seeds for the three bundled database servers include those classifications; other servers must be reviewed after live discovery.
• Calibration rejects tools whose policy requires approval because that route has no approval checkpoint. Approval-required calls belong in a governed workflow run; the admin-only playground remains an intentional break-glass surface.
• Aliases in CALIBER_MCP_REQUIRE_EXTERNAL_ISOLATION_FOR_ALIASES (default: prod) reject local stdio containment. A non-loopback HTTPS server or the deployment operator's explicitly attested, separately isolated sidecar is required. Local wrappers, including the recognized bubblewrap containment profile, never count as a production boundary.
• Deletion is not an unguarded hard delete: it is refused with 409 while an active deployment still references the server, and the full server definition is snapshotted into the delete audit row so the record is recoverable.
• Tool invocation runs under bounded timeouts.
• Connection configuration is validated against transport-specific minimums before use.

These controls express a specific trust posture. Local stdio remains a child process on the CALIBER host. Command allowlisting, sanitized environment, private cwd, and timeouts reduce ambient authority but do not prevent an allowed executable from reading absolute filesystem paths or opening network connections. The recognized bubblewrap profile blocks network and host writes but still exposes the host root read-only, so it remains containment and cannot pass production preflight. Operators must use a managed sidecar/container/VM or a separately isolated remote HTTPS MCP service for production. CALIBER trusts the operator to register the correct server endpoint, but not to bypass the policy checks that sit in front of it. Remote MCP servers are external execution dependencies that may fail or return tool errors, and the module is built to surface those outcomes rather than mask them. CALIBER owns the connection health reporting, the cached inventory, and the local tool policy, but it never owns the remote tool implementation itself.

The module reports its health and behavior through a compact set of signals that operators can read directly:

• The status, last_connected_at, and connection_error fields on the server row.
• The cached discovered tool catalog, which gives the UI a stable view to render.
• The per-tool effective policy returned by GET /tools.
• The calibration summaries stored on the server row.
• The invocation duration and gateway error text returned to the caller.

Underneath those signals, the gateway is designed to be reusable and guarded. It supports stdio, SSE, and streamable HTTP transports through one implementation. It exposes async APIs alongside sync wrappers for callers that run outside an event loop, notably the workflow runtime. And it can resolve the ${PYTHON} sentinel for first-party Python MCP servers, so that those servers launch under the same interpreter as CALIBER rather than relying on whatever python happens to be on the path.

The default compose deployment takes the stronger path for the bundled database integrations. It starts one streamable-HTTP process per database mode in a separate container, runs each as uid/gid 65532 with a read-only root, temporary /tmp, CPU/memory/process limits, all Linux capabilities dropped, no-new-privileges, and no published host port. The sidecars attach only to a dedicated Docker-internal network shared with CALIBER and PostgreSQL, which blocks ordinary external egress. The PostgreSQL, pgvector, and Apache AGE catalog presets point at those internal services. CALIBER allowlists and operator-attests only those three hostnames as managed sidecars. This is a shipped process/container boundary, but it is not end-to-end TLS or mutual isolation from the API/database peers on that internal network; deployments that change the compose topology must reassess that attestation.

The shipped database credential is deliberately a development default, not a production-safe data boundary. Unless CALIBER_MCP_POSTGRES_URL is overridden, the sidecars connect to the same caliber database with the same privileged role used by the CALIBER metadata store. A write or DDL tool can therefore damage control-plane state even though its process is container-isolated. Before production use, provision a separate database/schema and least-privilege role, set CALIBER_MCP_POSTGRES_URL to it, and review the seeded write/external-action policies. Process isolation and data authorization are independent controls.

The module is built to extend along clear lines. Its primary extension points are:

• Adding more curated server presets in the MCP UI.
• Adding richer auth-type handling and secret-source indirection.
• Expanding policy semantics beyond allow/deny and side-effect metadata.
• Adding first-party MCP servers under mcp_servers/*.

Those seams come with constraints worth keeping in mind. Discovery is cached, so the tool inventory can become stale until it is explicitly refreshed. Policy is local to CALIBER and is not negotiated with the remote server. Workflow runtime invocation is synchronous from the interpreter's perspective even though the underlying transport is async. And the quality and safety of a remote server are ultimately outside CALIBER's control. Rate limits are process-local rather than Redis-backed, and the bubblewrap profile is Linux-specific. Managed-sidecar classification is deployment-operator attested because the gateway cannot inspect another container's live security policy. The UI's execution readiness panel reports these controls and blockers; it must not be interpreted as a claim that ordinary local stdio is sandboxed or that the backing service's data privileges are least-privilege. Alias-rotation preflight resolves each subworkflow's deployed target and recursively inspects its MCP dependencies, using the same target-selection rules as runtime. An unresolved deployed child is a fail-closed blocker for alias rotation, and exhausting the 16-level traversal bound is always reported as unverifiable rather than silently treated as complete. Manifest-only callers without a database session remain root-only by design; nested runtime calls still pass through the central gateway and fail closed independently.

In sum, the MCP module is a mediated integration layer. It turns external MCP servers into policy-controlled, inspectable CALIBER runtime dependencies without scattering transport logic throughout the codebase.