Enterprise·
advanced
·15 min read·Sep 12, 2026
By Rad Tome·Lead AI Systems Architect

Deploying Remote MCP Servers on Kubernetes at Scale

Production guide to deploying containerized remote Model Context Protocol (MCP) servers on Kubernetes with Helm, JSON-RPC queue-based HPA, and secure Ingress.

kuberneteshelmmcpautoscalingcontainersdevops
Interactive Tool
1-Click Export

Generate & Validate Multi-Client MCP Config

One-click export with environment variables & path locators for Claude Desktop, Cursor, Windsurf, and OpenAI Codex CLI.

Open in Generator

Deploying Remote MCP Servers on Kubernetes at Scale

Moving Model Context Protocol (MCP) servers from local developer workstations running over stdio to shared enterprise Kubernetes clusters introduces unique orchestration challenges. Unlike standard stateless REST APIs, remote MCP servers communicate over long-lived Server-Sent Events (SSE) connections, process computationally expensive tool executions, and maintain in-memory protocol state during complex multi-step agent reasoning loops.

Deploying remote MCP servers in mission-critical environments requires declarative Helm packaging, hardened container security contexts, zero-downtime rolling updates with connection draining, and custom Horizontal Pod Autoscaling (HPA) based on active JSON-RPC queue depth rather than generic CPU or memory utilization.

This guide provides a production-tested blueprint for architecting, packaging, and operating remote MCP clusters on Kubernetes.

1. Architectural Topology: Remote MCP on Kubernetes

In a Kubernetes environment, client agents (such as Cursor, Windsurf, or autonomous agent clusters) connect via Ingress controllers that route streaming SSE traffic to backend pods. Each MCP server pod exposes an HTTP server terminating SSE streams and accepting POST requests for JSON-RPC 2.0 tool execution.

mermaid
graph TD
    Client[AI Client / Host IDE] -->|HTTPS Streamable SSE| Ingress[Ingress Controller: Traefik / NGINX]
    
    subgraph Kubernetes Namespace: ai-infrastructure
        Ingress -->|Sticky Session / Session Affinity| Service[mcp-database-service]
        Service --> Pod1[MCP Pod 1: Tool Worker]
        Service --> Pod2[MCP Pod 2: Tool Worker]
        Service --> Pod3[MCP Pod N: Tool Worker]
        
        Prometheus[Prometheus Exporter] -->|Scrape JSON-RPC Queue Depth| Service
        HPA[Custom Metrics HPA] -->|Scale In / Out| Service
    end
    
    Pod1 -->|Read Replica Pool| Postgres[(Enterprise Database)]
    Pod2 -->|Read Replica Pool| Postgres
    Pod3 -->|Read Replica Pool| Postgres

Core Operational Requirements

  • ▸Sticky Sessions: Because MCP SSE connections decouple the downstream event stream (GET /sse) from the upstream tool execution requests (POST /messages?sessionId=...), the Ingress must enforce sticky sessions via HTTP cookies or routing headers.
  • ▸Non-Blocking Streaming: Upstream Ingress controllers must disable response buffering to prevent JSON-RPC chunks from being held until buffer flush.
  • ▸Process Sandboxing: Pods must run as non-root users with read-only root filesystems, preventing compromised tools from modifying container binaries.

2. Remote Transport Wire Protocol: SSE Handshake

Remote MCP servers establish communication through the Server-Sent Events transport. Understanding the exact wire payloads is essential for configuring Kubernetes readiness probes and debugging proxy dropouts.

Step 1: Initial SSE Connection Request (GET /sse)

The client opens a persistent SSE connection:

http
GET /sse HTTP/1.1
Host: mcp-k8s.internal.enterprise.com
Authorization: Bearer <JWT_TOKEN>
Accept: text/event-stream
Cache-Control: no-cache

Step 2: Server Session Assignment

The remote MCP server immediately emits an endpoint event containing a unique session identifier and URI where subsequent JSON-RPC messages must be sent:

http
HTTP/1.1 200 OK
Content-Type: text/event-stream
Cache-Control: no-cache
Connection: keep-alive
X-Accel-Buffering: no

event: endpoint
data: /messages?sessionId=k8s-pod-9c8e1-4b72-91ef

Step 3: Tool Execution Post Request (POST /messages)

The client delivers JSON-RPC requests to the assigned endpoint:

http
POST /messages?sessionId=k8s-pod-9c8e1-4b72-91ef HTTP/1.1
Host: mcp-k8s.internal.enterprise.com
Content-Type: application/json

{
  "jsonrpc": "2.0",
  "id": "req-9812",
  "method": "tools/call",
  "params": {
    "name": "cluster_inspect_pod_logs",
    "arguments": {
      "namespace": "production",
      "pod_name": "checkout-service-7bb8",
      "lines": 50
    }
  }
}

Step 4: Streamed JSON-RPC Response

The server pushes the execution result back down the established SSE stream:

http
event: message
data: {"jsonrpc":"2.0","id":"req-9812","result":{"content":[{"type":"text","text":"[2026-09-12 18:22:01] INFO Connection pool initialized\n[2026-09-12 18:22:04] ERROR Connection timeout: payment-gateway:443"}]}}

3. Production Helm Chart Architecture

Below is a production-grade Helm chart layout for deploying remote MCP servers with hardened security profiles, probes, and resource bounds.

values.yaml

yaml
# Helm values for enterprise-mcp-server
replicaCount: 3

image:
  repository: registry.enterprise.internal/ai-infra/mcp-database-server
  pullPolicy: IfNotPresent
  tag: "1.4.2"

imagePullSecrets:
  - name: enterprise-registry-creds

securityContext:
  runAsNonRoot: true
  runAsUser: 10001
  runAsGroup: 10001
  readOnlyRootFilesystem: true
  allowPrivilegeEscalation: false
  capabilities:
    drop:
      - ALL

resources:
  limits:
    cpu: "2"
    memory: "2Gi"
  requests:
    cpu: "500m"
    memory: "512Mi"

env:
  NODE_ENV: "production"
  MCP_PORT: "8080"
  MAX_CONCURRENT_SESSIONS: "100"
  STATEMENT_TIMEOUT_MS: "5000"

service:
  type: ClusterIP
  port: 8080

ingress:
  enabled: true
  className: "nginx"
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-enterprise"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-buffering: "off"
    nginx.ingress.kubernetes.io/affinity: "cookie"
    nginx.ingress.kubernetes.io/session-cookie-name: "MCP_K8S_ROUTE"
    nginx.ingress.kubernetes.io/session-cookie-expires: "14400"
  hosts:
    - host: mcp-database.ai.enterprise.com
      paths:
        - path: /
          pathType: Prefix
  tls:
    - secretName: mcp-database-tls
      hosts:
        - mcp-database.ai.enterprise.com

autoscaling:
  enabled: true
  minReplicas: 3
  maxReplicas: 25
  targetQueueDepth: 15

templates/deployment.yaml

yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Release.Name }}
  labels:
    app.kubernetes.io/name: {{ .Release.Name }}
spec:
  replicas: {{ .Values.replicaCount }}
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  selector:
    matchLabels:
      app.kubernetes.io/name: {{ .Release.Name }}
  template:
    metadata:
      labels:
        app.kubernetes.io/name: {{ .Release.Name }}
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "8080"
        prometheus.io/path: "/metrics"
    spec:
      securityContext:
        fsGroup: 10001
      containers:
        - name: mcp-server
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          imagePullPolicy: {{ .Values.image.pullPolicy }}
          securityContext:
            {{- toYaml .Values.securityContext | nindent 12 }}
          ports:
            - name: http-mcp
              containerPort: 8080
              protocol: TCP
          env:
            {{- range $key, $val := .Values.env }}
            - name: {{ $key }}
              value: {{ $val | quote }}
            {{- end }}
          livenessProbe:
            httpGet:
              path: /healthz
              port: http-mcp
            initialDelaySeconds: 10
            periodSeconds: 15
          readinessProbe:
            httpGet:
              path: /ready
              port: http-mcp
            initialDelaySeconds: 5
            periodSeconds: 5
          resources:
            {{- toYaml .Values.resources | nindent 12 }}
          volumeMounts:
            - name: tmp-volume
              mountPath: /tmp
      volumes:
        - name: tmp-volume
          emptyDir: {}

4. Autoscaling on Active JSON-RPC Queue Depth

Standard CPU and memory metrics fail to reflect MCP server workload spikes. An MCP server awaiting a complex 30-second database query or vector search consumes negligible CPU while its thread pool and connection budget are fully saturated.

To solve this, instrument the MCP server with a Prometheus metric tracking active pending tool invocations: mcp_pending_tool_invocations.

Prometheus Instrumentation in TypeScript Server

typescript
import express from 'express';
import { Counter, Gauge, collectDefaultMetrics, Registry } from 'prom-client';

const register = new Registry();
collectDefaultMetrics({ register });

export const pendingToolGauge = new Gauge({
  name: 'mcp_pending_tool_invocations',
  help: 'Number of active JSON-RPC tool executions currently in flight.',
  labelNames: ['tool_name'],
  registers: [register]
});

export const toolInvocationDuration = new Counter({
  name: 'mcp_tool_execution_total',
  help: 'Total count of executed tools',
  labelNames: ['tool_name', 'status'],
  registers: [register]
});

const app = express();

// Expose Prometheus endpoint for Kubernetes scraping
app.get('/metrics', async (req, res) => {
  res.setHeader('Content-Type', register.contentType);
  res.send(await register.metrics());
});

Kubernetes HorizontalPodAutoscaler with Custom Metric

Using the Prometheus Adapter (custom.metrics.k8s.io), scale the deployment when the average pending tool queue per pod exceeds 15 concurrent calls:

yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: mcp-database-hpa
  namespace: ai-infrastructure
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: mcp-database-server
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Pods
      pods:
        metric:
          name: mcp_pending_tool_invocations
        target:
          type: AverageValue
          averageValue: "15"
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
        - type: Percent
          value: 100
          periodSeconds: 15
    scaleDown:
      stabilizationWindowSeconds: 300 # Prevent thrashing during LLM tool pauses
      policies:
        - type: Percent
          value: 20
          periodSeconds: 60

5. Graceful Connection Draining for SSE Streams

Because SSE connections are persistent, terminating a pod during a Kubernetes deployment rollout immediately severs active client agent sessions. Implement a preStop lifecycle hook and SIGTERM handler that allows in-flight tool calls to finish while signaling the client to reconnect.

yaml
# Inside Deployment.spec.template.spec.containers[0]
lifecycle:
  preStop:
    exec:
      command: ["/bin/sh", "-c", "sleep 10 && touch /tmp/draining"]

In the MCP server application logic:

typescript
let isShuttingDown = false;

process.on('SIGTERM', () => {
  console.log('SIGTERM received: initiating MCP graceful shutdown...');
  isShuttingDown = true;

  // 1. Notify connected SSE clients of server relocation
  for (const session of activeSseSessions.values()) {
    session.res.write(`event: notification\ndata: {"jsonrpc":"2.0","method":"notifications/server_draining","params":{"reconnect_seconds":3}}\n\n`);
  }

  // 2. Wait for active in-flight tool calls to resolve
  const drainInterval = setInterval(() => {
    if (activeToolExecutionsCount === 0) {
      clearInterval(drainInterval);
      server.close(() => {
        console.log('All MCP sessions drained. Process exiting cleanly.');
        process.exit(0);
      });
    }
  }, 500);
});

6. Verification and Health Check Runbook

Once deployed, verify cluster readiness and session routing using curl:

bash
# 1. Establish SSE Connection and verify session assignment
curl -i -N -H "Accept: text/event-stream" \
  https://mcp-database.ai.enterprise.com/sse

# Expected Response:
# HTTP/1.1 200 OK
# Content-Type: text/event-stream
# Set-Cookie: MCP_K8S_ROUTE=...
# event: endpoint
# data: /messages?sessionId=550e8400-e29b-41d4-a716-446655440000

# 2. Test JSON-RPC Tool Invocation over HTTP POST
curl -X POST -H "Content-Type: application/json" \
  -b "MCP_K8S_ROUTE=..." \
  -d '{
    "jsonrpc": "2.0",
    "id": 1,
    "method": "tools/call",
    "params": {
      "name": "health_check",
      "arguments": {}
    }
  }' \
  https://mcp-database.ai.enterprise.com/messages?sessionId=550e8400-e29b-41d4-a716-446655440000

Related Kubernetes & Enterprise Guides

Ready to Deploy?

Build your full agent toolstack in the Visual Generator

Combine Deploying Remote with databases, search APIs, and memory graphs in a single configuration file.

Customize in Generator

Deploying Remote MCP Servers on Kubernetes at Scale FAQ

What is the Deploying Remote MCP Servers on Kubernetes at Scale?

Production guide to deploying containerized remote Model Context Protocol (MCP) servers on Kubernetes with Helm, JSON-RPC queue-based HPA, and secure Ingress.

How do I configure Deploying Remote MCP Servers on Kubernetes at Scale in Claude Desktop or Cursor?

You can copy the configuration JSON from our guide or launch the interactive MCP Codex Config Generator at https://mcp-codex.com/generator to export valid configs in 1 click.

Can I use Deploying Remote MCP Servers on Kubernetes at Scale with the OpenAI Codex CLI?

Yes, OpenAI Codex CLI supports Model Context Protocol. You can add it directly to ~/.codex/config.toml or pass arguments to codex mcp add.

RT

Written by Rad Tome

Lead AI Systems Architect & Founder, MCP Codex

@RadTome

Specializing in Model Context Protocol (MCP) integrations, autonomous AI agent orchestration, and distributed developer toolchains. Researches and benchmarks production MCP client-server architectures across OpenAI Codex, Claude, and Cursor.

Editorial Integrity: All configurations, schemas, and commands verified against live GitHub repositories and tested in local sandbox runtimes.

Related Guides

Enterprise

Hardening MCP for SOC 2 and HIPAA Enterprise Workflows

Compliance architecture for Model Context Protocol (MCP): Client-side PII masking, immutable JSON-RPC audit logging, and zero-knowledge data pipelines for SOC 2 and HIPAA.

Enterprise

Sandboxing MCP Server Execution: Containers to MicroVMs

Isolating Model Context Protocol (MCP) server execution to neutralize arbitrary code execution, filesystem escapes, and credential exfiltration using Docker rootless, gVisor, and Firecracker microVMs.

Enterprise

Production Vector Search MCP Integration: Pinecone and Qdrant

Architecting enterprise vector search MCP tools with Pinecone, Qdrant, and Milvus: Hybrid BM25/dense vector retrieval, Reciprocal Rank Fusion (RRF), and token-budgeted formatting.