MCP Architecture for Production AI Systems

Model Context Protocol (MCP) makes it easier for an AI host to discover and invoke external capabilities through a common protocol. That interoperability is valuable, but it can hide an important fact: a protocol connection is a new production boundary. It can carry customer context, invoke tools, reach a third-party service, receive untrusted descriptions, and return results that influence an agent’s next action.

The right architectural response is not “connect every useful MCP server.” It is to decide which host, client, server, tool, identity, data, and approval boundary owns each capability—and to make those decisions visible in code and operations.

MCP Is a Boundary, Not a Trust Decision

MCP defines roles and messages; it does not turn an arbitrary server into a trusted extension of your application. The protocol specification itself cautions that tools can represent arbitrary code execution and that hosts should obtain explicit user consent before invocation. Treat an MCP server like any other integration: inspect its source or vendor, understand its retention and network behaviour, scope credentials, and monitor it in production.

The common trust mistake is transitive: “our agent is authorized, therefore every tool it discovers is authorized.” In reality, each hop needs a separate answer.

user → host application → MCP client → MCP server → downstream API/data store

Who is authenticated at each hop? What can that identity read or write? Which input crosses the boundary? What downstream tokens exist? Where are logs stored? What happens if the server returns adversarial instructions rather than ordinary data?

The MCP authorization specification (opens in new tab) provides a transport-level OAuth model for HTTP-based servers. It is a useful foundation, not the entire security design. Application authorization and domain rules still belong at the server and downstream API layers.

Draw the Architecture Before Adding Servers

Start with a concrete diagram and name the responsibilities:

A production host should maintain an allowlist of servers and, where possible, a run-specific allowlist of tools. The agent does not need every organizational connector for every task. Giving a writing workflow access to a payment server “in case it helps” enlarges the blast radius without improving the intended job.

Separate servers by data domain and risk. A read-only documentation server, a customer-support server, and a billing-mutation server have different owners, audit expectations, credentials, and outage consequences. One giant “company MCP” becomes a gateway that is hard to reason about, hard to test, and tempting to over-authorize.

Keep Tools Narrow and Servers Domain-Scoped

MCP exposes tools, resources, and prompts, but the same design discipline applies to every exposed capability. A tool should express a narrow domain action, clear arguments, meaningful result codes, and stable behaviour.

get_customer_order(order_id)
  → read-only, returns a redacted order view for an authorized caller
 
create_refund_review(order_id, reason_code)
  → creates an internal review request; never sends money directly

Avoid the all-purpose escape hatch:

run_sql(query)
call_internal_api(path, method, body)
execute_shell(command)

Such tools compress an entire authorization and validation model into one opaque model decision. If the underlying system truly needs flexible queries or commands, put them behind a constrained, separately governed workflow with audited templates and human approval rather than exposing raw power to an agent loop.

Tool descriptions are also untrusted-adjacent input. A malicious or compromised server may publish persuasive descriptions designed to influence the agent. The host must decide which servers and tools are available before model selection; do not rely on the model to recognize every deceptive instruction.

Authorize the Right Principal for the Right Resource

For HTTP-based protected MCP servers, the specification uses OAuth 2.1 concepts: the server is a resource server, the client acts for a resource owner, and tokens should be bound to the intended resource. The specification requires access tokens to be sent in authorization headers, not URLs, and requires servers to validate tokens issued for their own audience.

That solves an important transport problem, but a production server must still answer domain questions:

• Is this user allowed to see this customer record in this organization?
• Is the requested action within the user’s role and current workflow state?
• Is the requested scope narrower than the token’s broad organization permission?
• Does a high-impact action require explicit confirmation or a second approver?

Keep actor identity separate from model-generated arguments. The model can propose orderId: "ord_42"; the server derives the caller from a verified credential and checks whether that actor may touch ord_42. Never accept a model-supplied tenantId, userId, or role claim as the authorization source.

For local stdio servers, credentials may come from the environment instead of an HTTP authorization flow. That can be appropriate for a single-user developer environment, but it changes the threat model: the host process and server now share a local privilege boundary. Use a dedicated service identity with narrow permissions, not a developer’s broad production credential inherited by convenience.

Operate MCP as a Production Dependency

An MCP integration needs the same operational treatment as a database driver or payment gateway:

1. Version and pin the server implementation; review changes to tools, prompts, dependencies, and network destinations.
2. Record server identity, tool name/version, sanitized inputs, authorization decision, latency, result code, and trace id for every call.
3. Set timeouts, retry limits, circuit breakers, and a fallback or handoff path for server failure.
4. Test unavailable, slow, malformed, unauthorized, and over-broad results—not only successful calls.
5. Maintain a fast disable switch for one server or one high-risk tool without taking down the whole host.

Data governance is part of that inventory. OpenAI’s platform documentation notes that remote MCP servers are third-party services whose data retention and residency policies apply to the data sent to them. Classify what data can cross each connection, redact unnecessary fields, and avoid uploading broad conversation history merely because a server might find it useful.

Change management deserves special attention. A server can change its tool schema or its side effects independently of the host. Contract-test the expected capabilities at integration time, alert when discovery differs from the approved manifest, and stage upgrades in a non-production environment with representative permissions.

A Minimal Production Blueprint

For a first production integration, keep the shape boring:

Host policy: allow one named server and two read-only tools for this workflow
  ↓
MCP client: authenticate to the server; enforce tool allowlist and timeout
  ↓
MCP server: validate schema and caller; map to a scoped service account
  ↓
Domain API: enforce row-level/business authorization; return a redacted result
  ↓
Host trace: record call, result category, and user-visible outcome

Then add a write tool only after the read path has useful evaluations, reliable traces, a clear confirmation rule, and a rollback or compensating action. Interoperability lets systems grow quickly; deliberate boundaries let them grow safely.

Key Takeaways

• MCP is an interoperability layer, not an automatic trust relationship.
• Architect each connection across explicit host, client, server, downstream system, identity, data, and approval boundaries.
• Keep servers domain-scoped and tools narrow; do not expose generic raw query, shell, or API escape hatches to an agent.
• Validate token audience, derive actor identity from verified credentials, and enforce resource-level authorization at the server and downstream API.
• Operate each MCP server as a versioned production dependency with allowlists, traces, failure controls, data governance, and a disable switch.

Next, we will go deeper on agent authentication: identifying users, workloads, and agents distinctly so permissions, tokens, audit trails, and delegation remain understandable as systems grow.