Enterprise·
advanced
·16 min read·Sep 12, 2026
By Rad Tome·Lead AI Systems Architect

Building an Enterprise MCP Reverse Proxy and API Gateway

Architecting a high-throughput, multi-tenant MCP reverse proxy and API gateway with centralized token budgeting, per-tool rate limiting, and transport multiplexing.

mcpreverse-proxyapi-gatewayrate-limitingmulti-tenancyarchitecture
Interactive Tool
1-Click Export

Generate & Validate Multi-Client MCP Config

One-click export with environment variables & path locators for Claude Desktop, Cursor, Windsurf, and OpenAI Codex CLI.

Open in Generator

Building an Enterprise MCP Reverse Proxy and API Gateway

When scaling the Model Context Protocol (MCP) beyond localized developer desktops running isolated stdio processes, engineering teams immediately encounter the limits of point-to-point agent connections. Direct client-to-server connections in enterprise environments expose backend infrastructure to unauthenticated tool execution, unbounded LLM token consumption, cascading failure loops, and zero visibility into data exfiltration channels.

Solving this requires decoupling AI host clients (Cursor, Claude Code, Windsurf, Claude Desktop, and autonomous orchestration agents) from internal MCP server clusters using a dedicated Enterprise MCP Reverse Proxy and API Gateway.

This guide covers the production architecture, wire protocol dynamics, authentication propagation, centralized token budgeting, and concrete TypeScript implementation of an enterprise-grade MCP gateway.

1. Architectural Blueprint: The Enterprise MCP Gateway

In a zero-trust enterprise topology, clients never connect directly to target services. Instead, the MCP Gateway acts as an intelligent Layer 7 proxy specialized for stateful, asynchronous JSON-RPC 2.0 transport over Server-Sent Events (SSE) and Streamable HTTP.

mermaid
graph TD
    Client1[Cursor / Claude Code] -->|HTTPS / SSE + JWT| Gateway[Enterprise MCP Reverse Proxy]
    Client2[Enterprise LangGraph Agent] -->|HTTPS / SSE + mTLS| Gateway
    
    subgraph Gateway Engine
        Auth[AuthN & RBAC Policy Engine]
        Budget[Token Bucket & Rate Limiter]
        Registry[Dynamic Tool Registry & Filter]
        Router[JSON-RPC Session Multiplexer]
        Audit[Audit Logger & OTel Tracing]
    end
    
    Gateway --> Auth
    Auth --> Budget
    Budget --> Registry
    Registry --> Router
    Router --> Audit
    
    Router -->|Stdio Subprocess Pool| LocalMCP[Local Sandbox MCP Servers]
    Router -->|Internal mTLS / gRPC| K8sMCP[K8s Remote MCP Cluster]
    Router -->|Scoped HTTP/SSE| ThirdPartyMCP[External SaaS MCP Servers]

Core Responsibilities of the Gateway

  1. ▸Protocol Normalization & Multiplexing: Translating incoming stateless or SSE HTTP connections into pooled backend transports (internal SSE, WebSocket, or isolated containerized stdio pipes).
  2. ▸Identity & Scope Injection: Validating client identities (OIDC/JWT) and injecting cryptographically signed claims into downstream MCP tool calls.
  3. ▸Per-Tool Access Control & Token Budgeting: Restricting high-risk tools (execute_sql, deploy_service, write_file) to privileged roles while metering LLM context token usage.
  4. ▸Resilience & Circuit Breaking: Trapping backend timeouts and schema mismatches before they break client agent runtimes.

2. Wire Protocol Dynamics: JSON-RPC 2.0 Negotiation

The Model Context Protocol operates strictly over JSON-RPC 2.0. The gateway intercepts, inspects, mutates, and routes every frame during the three lifecycle phases: Initialization, Tool Discovery, and Tool Execution.

Phase 1: Client Handshake (initialize)

The incoming client handshake initiates the session and declares client capabilities:

json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "initialize",
  "params": {
    "protocolVersion": "2024-11-05",
    "capabilities": {
      "roots": {
        "listChanged": true
      },
      "sampling": {}
    },
    "clientInfo": {
      "name": "Cursor-Enterprise",
      "version": "0.45.2"
    }
  }
}

The gateway intercepts this payload, assigns an enterprise session identifier (mcp-session-c9f28a), attaches client rate-limit tier metadata, and returns negotiated capabilities supported across downstream federated servers:

json
{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "protocolVersion": "2024-11-05",
    "capabilities": {
      "tools": {
        "listChanged": true
      },
      "resources": {
        "subscribe": false,
        "listChanged": false
      }
    },
    "serverInfo": {
      "name": "Enterprise-MCP-Gateway",
      "version": "2.4.0"
    }
  }
}

Phase 2: Role-Based Tool Filtering (tools/list)

When the host requests available tools, the gateway must not expose every backend tool indiscriminately. It filters the catalog according to tenant RBAC permissions:

json
{
  "jsonrpc": "2.0",
  "id": 2,
  "method": "tools/list",
  "params": {}
}

The gateway aggregates schemas from 10+ backend servers, strips unauthorized tools, and prefixes tool names with namespace identifiers to prevent collisions (e.g., db_primary__query_read vs finance__query_read):

json
{
  "jsonrpc": "2.0",
  "id": 2,
  "result": {
    "tools": [
      {
        "name": "db_primary__select_query",
        "description": "Execute read-only SQL queries against the read replica.",
        "inputSchema": {
          "type": "object",
          "properties": {
            "query": {
              "type": "string",
              "description": "Standard ANSI SQL SELECT statement."
            }
          },
          "required": ["query"]
        }
      }
    ]
  }
}

Phase 3: Monitored Tool Execution (tools/call)

When an agent invokes a tool, the gateway checks real-time rate limits and token budgets before forwarding:

json
{
  "jsonrpc": "2.0",
  "id": 3,
  "method": "tools/call",
  "params": {
    "name": "db_primary__select_query",
    "arguments": {
      "query": "SELECT user_id, email, organization_id FROM users WHERE status = 'active' LIMIT 100;"
    }
  }
}

If tenant limits are violated, the gateway drops execution and returns a typed JSON-RPC error payload immediately, preventing downstream saturation:

json
{
  "jsonrpc": "2.0",
  "id": 3,
  "error": {
    "code": -32001,
    "message": "Resource budget exceeded: Tenant 'org_acme' has exhausted its hourly LLM tool invocation quota (1,000 calls/hr).",
    "data": {
      "tenant_id": "org_acme",
      "retry_after_seconds": 312,
      "policy": "enterprise-standard-budget"
    }
  }
}

3. Centralized Token Budgeting & Leaky Bucket Rate Limiting

Unlike traditional REST APIs measured in simple Requests Per Second (RPS), MCP gateways must handle two distinct dimensions:

  1. ▸Invocation Frequency (RPS): How often an agent calls specific tools.
  2. ▸Context Token Load (Payload Volume): The estimated token size of tool inputs and outputs injected back into the LLM context window.
mermaid
flowchart LR
    Request[tools/call Request] --> RedisCheck{Check Redis Token Bucket}
    RedisCheck -->|Quota Available| DeductTokens[Deduct Invocation Cost]
    DeductTokens --> RouteBackend[Forward to Backend MCP Server]
    RouteBackend --> MeasureOutput[Measure Response Byte/Token Count]
    MeasureOutput --> BillTenant[Record Audit Metric]
    RedisCheck -->|Quota Exhausted| Reject[-32001 QuotaExceeded Error]

Tiered Rate-Limiting Policy Matrix

Tool CategoryPermitted RolesDefault Limit (Calls/Min)Max Output Token CapTimeout Ceiling
Introspection (describe_schema, list_repos)ReadOnly, Dev, Admin1204,000 tokens3,000 ms
Data Query (select_query, read_logs)Dev, Admin3016,000 tokens8,000 ms
State Mutation (create_pr, restart_pod)SRE, LeadEngineer52,000 tokens15,000 ms
Dangerous Execution (apply_terraform, drop_table)BreakGlassAdmin1 (Requires 2FA)1,000 tokens30,000 ms

4. Production Gateway Implementation: TypeScript & Node.js

Below is an enterprise MCP Gateway built with Fastify and TypeScript. It implements JWT verification, per-tenant Redis rate limiting, session-to-backend routing, and full JSON-RPC validation.

typescript
import Fastify, { FastifyRequest, FastifyReply } from 'fastify';
import Redis from 'ioredis';
import jwt from 'jsonwebtoken';
import { v4 as uuidv4 } from 'uuid';

const app = Fastify({ logger: true });
const redis = new Redis(process.env.REDIS_URL || 'redis://');
const JWT_SECRET = process.env.JWT_SECRET || 'super-secret-enterprise-key-2026';

interface JsonRpcRequest {
  jsonrpc: '2.0';
  id: string | number;
  method: string;
  params?: Record<string, any>;
}

interface TenantContext {
  tenantId: string;
  roles: string[];
  tier: 'free' | 'pro' | 'enterprise';
}

// 1. Authentication Middleware
app.decorateRequest('tenant', null);

app.addHook('preHandler', async (request: FastifyRequest, reply: FastifyReply) => {
  const authHeader = request.headers.authorization;
  if (!authHeader || !authHeader.startsWith('Bearer ')) {
    reply.status(401).send({
      jsonrpc: '2.0',
      id: null,
      error: { code: -32650, message: 'Unauthorized: Missing or invalid Bearer token' }
    });
    return;
  }

  const token = authHeader.substring(7);
  try {
    const decoded = jwt.verify(token, JWT_SECRET) as TenantContext;
    (request as any).tenant = decoded;
  } catch (err) {
    reply.status(403).send({
      jsonrpc: '2.0',
      id: null,
      error: { code: -32651, message: 'Forbidden: Invalid JWT signature or token expired' }
    });
  }
});

// 2. Token Budget & Rate Limiting Enforcement
async function enforceToolBudget(tenantId: string, toolName: string): Promise<boolean> {
  const key = `ratelimit:${tenantId}:${toolName}`;
  const currentUsage = await redis.incr(key);
  
  if (currentUsage === 1) {
    await redis.expire(key, 60); // 1-minute window
  }

  const limit = toolName.includes('write') || toolName.includes('execute') ? 10 : 60;
  return currentUsage <= limit;
}

// 3. Routing Engine & Session Forwarding
app.post('/v1/mcp', async (request: FastifyRequest, reply: FastifyReply) => {
  const body = request.body as JsonRpcRequest;
  const tenant = (request as any).tenant as TenantContext;

  if (!body.jsonrpc || body.jsonrpc !== '2.0' || !body.method) {
    return reply.status(400).send({
      jsonrpc: '2.0',
      id: body?.id || null,
      error: { code: -32600, message: 'Invalid JSON-RPC 2.0 Request' }
    });
  }

  // Handle Lifecycle Methods
  if (body.method === 'initialize') {
    return reply.send({
      jsonrpc: '2.0',
      id: body.id,
      result: {
        protocolVersion: '2024-11-05',
        capabilities: { tools: { listChanged: true } },
        serverInfo: { name: 'Enterprise-MCP-Gateway', version: '2.4.0' }
      }
    });
  }

  // Handle Tool Calling with Rate Limiting
  if (body.method === 'tools/call') {
    const toolName = body.params?.name;
    if (!toolName) {
      return reply.send({
        jsonrpc: '2.0',
        id: body.id,
        error: { code: -32602, message: 'Missing tool name in params' }
      });
    }

    const isAllowed = await enforceToolBudget(tenant.tenantId, toolName);
    if (!isAllowed) {
      return reply.status(429).send({
        jsonrpc: '2.0',
        id: body.id,
        error: {
          code: -32001,
          message: `Rate limit exceeded for tool '${toolName}'. Retry in next window.`,
          data: { tenantId: tenant.tenantId, tool: toolName }
        }
      });
    }

    // Proxy request downstream to targeted microservice MCP server
    try {
      const backendUrl = resolveBackendForTool(toolName);
      const upstreamResponse = await fetch(backendUrl, {
        method: 'POST',
        headers: {
          'Content-Type': 'application/json',
          'X-Tenant-Id': tenant.tenantId,
          'X-Correlation-Id': uuidv4()
        },
        body: JSON.stringify(body)
      });

      const data = await upstreamResponse.json();
      return reply.send(data);
    } catch (err: any) {
      return reply.status(502).send({
        jsonrpc: '2.0',
        id: body.id,
        error: { code: -32603, message: `Upstream MCP backend unreachable: ${err.message}` }
      });
    }
  }

  return reply.send({
    jsonrpc: '2.0',
    id: body.id,
    error: { code: -32601, message: `Method '${body.method}' not implemented by gateway` }
  });
});

function resolveBackendForTool(toolName: string): string {
  if (toolName.startsWith('db_')) return process.env.DB_MCP_SERVICE_URL || 'https://mcp-db:8080/mcp';
  if (toolName.startsWith('k8s_')) return process.env.K8S_MCP_SERVICE_URL || 'https://mcp-k8s:8080/mcp';
  return process.env.DEFAULT_MCP_SERVICE_URL || 'https://mcp-core:8080/mcp';
}

app.listen({ port: 8080, host: '0.0.0.0' }, (err) => {
  if (err) {
    app.log.error(err);
    process.exit(1);
  }
  console.log('Enterprise MCP Gateway listening on port 8080');
});

5. Gateway Ingress Routing with NGINX / Envoy

When terminating thousands of persistent SSE connections from AI IDEs, NGINX must be configured to prevent buffer stalls, enable immediate HTTP/2 server push, and disable proxy buffering.

nginx
# /etc/nginx/conf.d/mcp-gateway.conf
upstream mcp_gateway_cluster {
    server 10.0.1.10:8080 max_fails=3 fail_timeout=10s;
    server 10.0.1.11:8080 max_fails=3 fail_timeout=10s;
    keepalive 64;
}

server {
    listen 443 ssl http2;
    server_name mcp-gateway.internal.enterprise.com;

    ssl_certificate /etc/ssl/certs/mcp-gateway.crt;
    ssl_certificate_key /etc/ssl/private/mcp-gateway.key;
    ssl_protocols TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;

    # Maximum payload size for large vector/context tools
    client_max_body_size 32M;

    location /v1/mcp {
        proxy_pass https://mcp_gateway_cluster;
        proxy_http_version 1.1;
        
        # Mandatory settings for MCP Server-Sent Events (SSE) streaming
        proxy_set_header Connection '';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Disable response buffering so tool streams emit immediately
        proxy_buffering off;
        proxy_cache off;
        proxy_read_timeout 300s;
        proxy_send_timeout 300s;
        chunked_transfer_encoding on;
    }
}

6. Enterprise Client Configuration (config.toml & mcp.json)

To route client traffic through the gateway rather than launching local unmanaged processes, configure clients using remote Streamable HTTP/SSE endpoints.

Windsurf & Claude Desktop Configuration (mcp.json)

json
{
  "mcpServers": {
    "enterprise-gateway": {
      "url": "https://mcp-gateway.internal.enterprise.com/v1/mcp",
      "headers": {
        "Authorization": "Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
        "X-Developer-Email": "lead.architect@enterprise.com"
      }
    }
  }
}

OpenAI Codex CLI Configuration (config.toml)

toml
[mcp_servers.enterprise_gateway]
url = "https://mcp-gateway.internal.enterprise.com/v1/mcp"
timeout_seconds = 60
bearer_token = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."

[mcp_servers.enterprise_gateway.tool_overrides]
auto_approve = ["db_primary__select_query"]
require_confirmation = ["k8s_cluster__restart_deployment"]

7. Security Hardening & Zero-Trust Checklist

Before promoting your MCP Gateway to production, audit your implementation against these critical controls:

  1. ▸Deny-by-Default Tool Exposure: New backend MCP servers must require explicit gateway configuration before tools appear in tools/list.
  2. ▸Context Window Token Throttling: Cap total aggregate payload sizes returned by tools/call to prevent a rogue query from saturating the agent's context window.
  3. ▸Correlation ID Tracing: Propagate X-Correlation-Id across every JSON-RPC handshake for auditability.
  4. ▸Header Sanitization: Strip internal auth tokens before forwarding results back to the LLM client.

Related Production Guides

Ready to Deploy?

Build your full agent toolstack in the Visual Generator

Combine Building an Enterprise MCP Reverse Proxy and API Gateway with databases, search APIs, and memory graphs in a single configuration file.

Customize in Generator

Building an Enterprise MCP Reverse Proxy and API Gateway FAQ

What is the Building an Enterprise MCP Reverse Proxy and API Gateway?

Architecting a high-throughput, multi-tenant MCP reverse proxy and API gateway with centralized token budgeting, per-tool rate limiting, and transport multiplexing.

How do I configure Building an Enterprise MCP Reverse Proxy and API Gateway in Claude Desktop or Cursor?

You can copy the configuration JSON from our guide or launch the interactive MCP Codex Config Generator at https://mcp-codex.com/generator to export valid configs in 1 click.

Can I use Building an Enterprise MCP Reverse Proxy and API Gateway with the OpenAI Codex CLI?

Yes, OpenAI Codex CLI supports Model Context Protocol. You can add it directly to ~/.codex/config.toml or pass arguments to codex mcp add.

RT

Written by Rad Tome

Lead AI Systems Architect & Founder, MCP Codex

@RadTome

Specializing in Model Context Protocol (MCP) integrations, autonomous AI agent orchestration, and distributed developer toolchains. Researches and benchmarks production MCP client-server architectures across OpenAI Codex, Claude, and Cursor.

Editorial Integrity: All configurations, schemas, and commands verified against live GitHub repositories and tested in local sandbox runtimes.

Related Guides

Enterprise

Deploying Remote MCP Servers on Kubernetes at Scale

Production guide to deploying containerized remote Model Context Protocol (MCP) servers on Kubernetes with Helm, JSON-RPC queue-based HPA, and secure Ingress.

Enterprise

Hardening MCP for SOC 2 and HIPAA Enterprise Workflows

Compliance architecture for Model Context Protocol (MCP): Client-side PII masking, immutable JSON-RPC audit logging, and zero-knowledge data pipelines for SOC 2 and HIPAA.

Enterprise

Sandboxing MCP Server Execution: Containers to MicroVMs

Isolating Model Context Protocol (MCP) server execution to neutralize arbitrary code execution, filesystem escapes, and credential exfiltration using Docker rootless, gVisor, and Firecracker microVMs.