AI Frameworks·
advanced
·13 min read·Sep 24, 2026·Updated Nov 12, 2026
·Lead AI Systems Architect·Verification Charter

MCP Context Caching and Prompt Compression

Learn how to slash LLM costs and latency by combining Anthropic prompt caching, OpenAI prefix caching, and AST-based Model Context Protocol schema compression.

context-cachingprompt-compressiontoken-optimizationmcplatencycost-reductionanthropic
Interactive Tool
1-Click Export

Generate & Validate Multi-Client MCP Config

One-click export with environment variables & path locators for Claude Desktop, Cursor, Windsurf, and OpenAI Codex CLI.

Open in Generator

MCP Context Caching and Prompt Compression

As enterprise AI agents expand from toy single-server integrations into production workflows connecting eight to fifteen Model Context Protocol (MCP) servers simultaneously (such as PostgreSQL, GitHub, Slack, Jira, AWS, Kubernetes, and Sentry), tool definitions alone can consume between 20,000 and 60,000 input tokens. In a standard twenty-turn diagnostic or coding session, repeatedly transmitting and recomputing attention matrices over these massive JSON schemas inflates time-to-first-token (TTFT) latency to five or eight seconds per turn and rapidly exhausts token budgets.

By combining provider-level KV-cache reuse (Anthropic Ephemeral Cache Breakpoints, OpenAI automatic prefix caching, and Google Gemini context caching) with AST-level schema compression and deterministic catalog sorting, engineering teams can slash input token overhead by up to eighty percent while cutting response latency by more than seventy percent.

This technical guide details the Transformer KV-cache mechanics governing tool prefix reuse, provides production-tested compression algorithms, and demonstrates multi-host configurations across Claude Desktop, Cursor, and the OpenAI Codex CLI.


1. Problem Statement and Architecture

When an autonomous agent connects to multiple MCP servers, the host application queries each process using the standard JSON-RPC 2.0 tools/list endpoint during startup. The client aggregates every returned tool definition into the system prompt or function declarations array submitted to the foundation model.

code
Without Prefix Caching (Every Turn Recomputes Full Context):
┌────────────────────────────────────────────────────────┐
│ Turn 1: 35,000 Tool Schema Tokens + 250 User Tokens    │
├────────────────────────────────────────────────────────┤
│ Turn 2: 35,000 Tool Schema Tokens + 1,400 Turn Tokens  │
├────────────────────────────────────────────────────────┤
│ Turn 3: 35,000 Tool Schema Tokens + 3,100 Turn Tokens  │
├────────────────────────────────────────────────────────┤
│ ... Turn 20: 35,000 Tool Tokens + 24,000 Prior History │
└────────────────────────────────────────────────────────┘
Cumulative Input Attention: ~1,100,000 tokens evaluated!

With Optimized MCP Prefix Caching:
┌────────────────────────────────────────────────────────┐
│ [Static System Prompt] (Sorted & Stripped Schemas)    │ -> KV-Cache Write (Turn 1)
│ [Cache Breakpoint: Ephemeral Marker]                  │
├────────────────────────────────────────────────────────┤
│ Turn 2..20: Read from Memory (90% Cost / 75% TTFT Drop)│ -> KV-Cache Read Hit
└────────────────────────────────────────────────────────┘

In modern Transformer architectures, the Key-Value (KV) cache stores intermediate activation states from self-attention layers so previous tokens do not require full parallel recomputation. When a host model processes an incoming prompt, it computes attention across all tokens from left to right. If the initial sequence of tokens exactly matches a previously processed prefix byte-for-byte, the inference cluster loads the precomputed KV tensors directly from GPU High Bandwidth Memory (HBM).

However, traditional MCP client implementations routinely break prefix cache reuse due to two subtle anti-patterns:

  1. ▸Non-deterministic tool enumeration: When multiple MCP servers launch concurrently over stdio or SSE transports, their responses arrive in arbitrary network order. If tool schemas are appended in arrival order, the token sequence changes between turns, invalidating the entire GPU KV cache.
  2. ▸Cosmetic schema bloat: Standard JSON Schema definitions generated by tool frameworks (such as Pydantic, Zod, or JSON Schema Draft 7) include extensive cosmetic metadata properties (title, $schema, examples, default, and deep nesting) that add zero semantic value to model reasoning while consuming tens of thousands of tokens.

By stabilizing the prefix order and stripping non-essential schema tokens before invoking LLM inference, the host maximizes KV-cache hits across all conversation turns.


2. Prerequisites and Environment Matrix

Before implementing prefix caching and compression middleware, verify your local development environment satisfies the runtime requirements:

ComponentMinimum VersionRecommendedNotes
Node.js>= 20.0.0 LTSv22.12.0 LTSRequired for ESM stream processing
Python>= 3.11.03.12.4Required for fast dictionary filtering
@modelcontextprotocol/sdk1.6.01.7.0Core TypeScript MCP protocol library
Anthropic SDK (Python)0.40.00.45.0Required for cache_control ephemeral syntax
OpenAI SDK (Python)1.50.01.58.0Automatic prefix caching enabled by default

Ensure your target model supports prompt caching:

  • ▸Anthropic: claude-3-7-sonnet-20250219, claude-3-5-sonnet-20241022, and claude-3-5-haiku-20241022 (minimum cacheable prompt length is 1,024 tokens for Sonnet and 2,048 tokens for Haiku).
  • ▸OpenAI: gpt-4o, gpt-4o-mini, and o1 (automatically caches prompts exceeding 1,024 tokens).
  • ▸Google: gemini-2.5-flash and gemini-2.5-pro (explicit context caching available via Google GenAI SDK).

3. Multi-Host Configuration Snippets

To ensure your desktop and CLI clients leverage caching and preserve predictable tool schemas, configure host environments with static definitions and pinned transports.

Claude Desktop Configuration

Claude Desktop automatically caches the system prompt and tool definitions when the combined prefix exceeds the model's threshold. Place pinned servers in your operating system's configuration file:

json
{
  "mcpServers": {
    "filesystem": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-filesystem", "/workspace"]
    },
    "postgres": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-postgres", "postgresql:///analytics"]
    },
    "github": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-github"],
      "env": {
        "GITHUB_PERSONAL_ACCESS_TOKEN": "ghp_EXAMPLE_TOKEN_FOR_AUTHENTICATION"
      }
    }
  }
}

Cursor IDE Configuration (.cursor/mcp.json)

When working inside Cursor, declare MCP servers with explicit environment isolation to prevent dynamic token drift:

json
{
  "mcpServers": {
    "filesystem": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-filesystem", "./src"]
    },
    "postgres": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-postgres", "postgresql:///analytics"]
    }
  }
}

OpenAI Codex CLI Configuration (~/.codex/config.toml)

For developers leveraging the OpenAI Codex CLI, register MCP tool providers with consistent name flags:

toml
[mcp.servers.filesystem]
command = "npx"
args = ["-y", "@modelcontextprotocol/server-filesystem", "/workspace"]

[mcp.servers.postgres]
command = "npx"
args = ["-y", "@modelcontextprotocol/server-postgres", "postgresql:///analytics"]

4. Step-by-Step Tool Discovery, Compression, and Caching

Step 1: Discover Tools via JSON-RPC 2.0

When the agent boots, it sends a tools/list request to each running MCP server:

json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/list",
  "params": {}
}

The server responds with comprehensive schema descriptions that include unnecessary verbose metadata:

json
{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "tools": [
      {
        "name": "query_database",
        "description": "Execute arbitrary read-only SQL queries against the analytical PostgreSQL cluster and return row records in structured format.",
        "inputSchema": {
          "$schema": "https://json-schema.org/draft-07/schema#",
          "title": "QueryDatabaseParameters",
          "type": "object",
          "properties": {
            "sql": {
              "type": "string",
              "title": "SQL Query",
              "description": "A valid SQL SELECT statement targeting public reporting views.",
              "examples": ["SELECT id, email FROM users LIMIT 10;"]
            },
            "timeout_ms": {
              "type": "integer",
              "title": "Query Timeout",
              "description": "Maximum execution time in milliseconds before query cancellation.",
              "default": 5000
            }
          },
          "required": ["sql"]
        }
      }
    ]
  }
}

Step 2: AST Schema Compression Algorithm

To eliminate token waste, run every returned tool schema through an AST cleaner that strips cosmetic JSON Schema keys while preserving parameter validation boundaries:

typescript
// mcp-schema-compressor.ts
export interface McpRawTool {
  name: string;
  description?: string;
  inputSchema?: Record<string, any>;
}

export interface CompressedTool {
  name: string;
  description: string;
  input_schema: Record<string, any>;
}

const DISCARD_KEYS = new Set([
  "$schema",
  "title",
  "examples",
  "default",
  "additionalProperties"
]);

function cleanSchemaNode(node: any): any {
  if (!node || typeof node !== "object") {
    return node;
  }
  if (Array.isArray(node)) {
    return node.map(cleanSchemaNode);
  }

  const cleaned: Record<string, any> = {};
  for (const [key, value] of Object.entries(node)) {
    if (DISCARD_KEYS.has(key)) {
      continue;
    }
    // Truncate overly verbose property descriptions to 80 characters
    if (key === "description" && typeof value === "string" && value.length > 80) {
      cleaned[key] = value.substring(0, 77) + "...";
      continue;
    }
    cleaned[key] = cleanSchemaNode(value);
  }
  return cleaned;
}

export function compressAndSortTools(tools: McpRawTool[]): CompressedTool[] {
  // 1. Sort tools deterministically by name to ensure stable token sequences
  const sorted = [...tools].sort((a, b) => a.name.localeCompare(b.name));

  // 2. Compress descriptions and strip JSON Schema metadata bloat
  return sorted.map((tool) => ({
    name: tool.name,
    description: (tool.description || "").substring(0, 140),
    input_schema: cleanSchemaNode(tool.inputSchema || { type: "object", properties: {} })
  }));
}

Step 3: Injecting Anthropic Ephemeral Cache Breakpoints

When calling the Anthropic Messages API, mark the final tool in the deterministically sorted array with the cache_control parameter:

python
# cached_mcp_orchestrator.py
import anthropic
from typing import List, Dict, Any

def build_cached_tool_payload(compressed_tools: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
    """
    Format compressed tools and place an ephemeral cache breakpoint on the
    final tool in the sorted catalog.
    """
    payload = []
    total_tools = len(compressed_tools)

    for idx, tool in enumerate(compressed_tools):
        entry = {
            "name": tool["name"],
            "description": tool["description"],
            "input_schema": tool["input_schema"]
        }
        # Inject cache breakpoint at the boundary of static tool declarations
        if idx == total_tools - 1:
            entry["cache_control"] = {"type": "ephemeral"}
        payload.append(entry)

    return payload

client = anthropic.Anthropic()

def execute_cached_turn(conversation_history: List[Dict[str, Any]], tools: List[Dict[str, Any]]):
    formatted_tools = build_cached_tool_payload(tools)
    
    response = client.messages.create(
        model="claude-3-7-sonnet-20250219",
        max_tokens=2048,
        tools=formatted_tools,
        messages=conversation_history
    )
    
    usage = response.usage
    cached_read = getattr(usage, "cache_read_input_tokens", 0)
    cached_write = getattr(usage, "cache_creation_input_tokens", 0)
    print(f"[Cache Metrics] Write Tokens: {cached_write} | Read Hit Tokens: {cached_read}")
    return response

On Turn 1, cache_creation_input_tokens reports the size of the combined system prompt and compressed tool declarations. On Turn 2 through Turn 20, cache_read_input_tokens captures the full prefix size while billed at a ninety percent discount, yielding near-instantaneous first-token responses.


5. Common Errors and Production Troubleshooting

Error 1: 0% Cache Read Hits Despite Identical Tools

  • ▸Symptom: cache_read_input_tokens remains zero across all conversation turns, incurring full pricing on every request.
  • ▸Root Cause: Tool schemas are aggregated dynamically from asynchronous background processes without sorting. If Server A finishes before Server B on Turn 1, but Server B finishes before Server A on Turn 2, the serialized JSON prompt prefix differs, forcing the foundation model to discard the cached KV tensors.
  • ▸Resolution: Always pipe tool arrays through a strict lexical sort function tools.sort((a, b) => a.name.localeCompare(b.name)) before generating API request frames.

Error 2: 400 invalid_request_error: cache_control limit exceeded

  • ▸Symptom: Anthropic API returns HTTP status code 400 with message stating that maximum cache breakpoints have been exceeded.
  • ▸Root Cause: The Anthropic API restricts each request to a maximum of four cache_control breakpoints. Developers mistakenly apply cache_control: {"type": "ephemeral"} to every tool definition in a loop rather than strictly to the final tool in the array.
  • ▸Resolution: Ensure your code checks if idx == len(tools) - 1: so exactly one breakpoint marks the entire tool list prefix.

Error 3: Windows CRLF Invalidation in Cross-Platform Environments

  • ▸Symptom: Cache hits function reliably on Linux CI/CD runners but consistently drop to zero when running locally on Windows workstations.
  • ▸Root Cause: System prompts and JSON serialization libraries on Windows format multiline string descriptions with \r\n line breaks, whereas Linux and macOS use \n. Because tokenizers assign distinct integer token IDs to carriage returns, cross-platform cache states fail to align.
  • ▸Resolution: Normalize all prompt strings and serialized schemas using .replace(/\r\n/g, '\n') prior to dispatching HTTP or stdio requests.

6. External Citations and Security Guardrails

Official Specifications and References

Security and Isolation Guardrails

When engineering cached tool prefixes, enforce strict tenant boundary controls:

  1. ▸Never cache tenant-specific identifiers in global prefixes: If multiple user sessions share a centralized agent gateway, do not include organization IDs, personal access tokens, or user names in the cached tool description block. Dynamic context belongs in user turns after the cache boundary.
  2. ▸Sanitize dynamic tool parameters: Tools that inject runtime metadata into schema titles or descriptions invalidate caching across instances. Keep tool definitions static and stateless.
Ready to Deploy?

Build your full agent toolstack in the Visual Generator

Combine verified MCP servers with databases, search APIs, and memory graphs in a single configuration file.

Customize in Generator
Developer Verification & Feedback
0 reports

Did this setup guide work with your AI host?

Help test this configuration across Claude, Cursor, and Codex CLI. Be the first to report:

RT✓

Written by Rad Tome

Lead AI Systems Architect & Maintainer, MCP Codex

Specializing in Model Context Protocol (MCP) integrations, autonomous AI agent orchestration, and distributed developer toolchains. Researches and benchmarks production MCP client-server architectures across OpenAI Codex, Claude, and Cursor.

Editorial Integrity: Package names verified via npm registry API. Repositories verified via GitHub REST API.
Verification Charter →

Related Guides