MCP Context Caching and Prompt Compression
Learn how to slash LLM costs and latency by combining Anthropic prompt caching, OpenAI prefix caching, and AST-based Model Context Protocol schema compression.
Generate & Validate Multi-Client MCP Config
One-click export with environment variables & path locators for Claude Desktop, Cursor, Windsurf, and OpenAI Codex CLI.
MCP Context Caching and Prompt Compression
As enterprise AI agents expand from toy single-server integrations into production workflows connecting eight to fifteen Model Context Protocol (MCP) servers simultaneously (such as PostgreSQL, GitHub, Slack, Jira, AWS, Kubernetes, and Sentry), tool definitions alone can consume between 20,000 and 60,000 input tokens. In a standard twenty-turn diagnostic or coding session, repeatedly transmitting and recomputing attention matrices over these massive JSON schemas inflates time-to-first-token (TTFT) latency to five or eight seconds per turn and rapidly exhausts token budgets.
By combining provider-level KV-cache reuse (Anthropic Ephemeral Cache Breakpoints, OpenAI automatic prefix caching, and Google Gemini context caching) with AST-level schema compression and deterministic catalog sorting, engineering teams can slash input token overhead by up to eighty percent while cutting response latency by more than seventy percent.
This technical guide details the Transformer KV-cache mechanics governing tool prefix reuse, provides production-tested compression algorithms, and demonstrates multi-host configurations across Claude Desktop, Cursor, and the OpenAI Codex CLI.
1. Problem Statement and Architecture
When an autonomous agent connects to multiple MCP servers, the host application queries each process using the standard JSON-RPC 2.0 tools/list endpoint during startup. The client aggregates every returned tool definition into the system prompt or function declarations array submitted to the foundation model.
Without Prefix Caching (Every Turn Recomputes Full Context):
┌────────────────────────────────────────────────────────┐
│ Turn 1: 35,000 Tool Schema Tokens + 250 User Tokens │
├────────────────────────────────────────────────────────┤
│ Turn 2: 35,000 Tool Schema Tokens + 1,400 Turn Tokens │
├────────────────────────────────────────────────────────┤
│ Turn 3: 35,000 Tool Schema Tokens + 3,100 Turn Tokens │
├────────────────────────────────────────────────────────┤
│ ... Turn 20: 35,000 Tool Tokens + 24,000 Prior History │
└────────────────────────────────────────────────────────┘
Cumulative Input Attention: ~1,100,000 tokens evaluated!
With Optimized MCP Prefix Caching:
┌────────────────────────────────────────────────────────┐
│ [Static System Prompt] (Sorted & Stripped Schemas) │ -> KV-Cache Write (Turn 1)
│ [Cache Breakpoint: Ephemeral Marker] │
├────────────────────────────────────────────────────────┤
│ Turn 2..20: Read from Memory (90% Cost / 75% TTFT Drop)│ -> KV-Cache Read Hit
└────────────────────────────────────────────────────────┘In modern Transformer architectures, the Key-Value (KV) cache stores intermediate activation states from self-attention layers so previous tokens do not require full parallel recomputation. When a host model processes an incoming prompt, it computes attention across all tokens from left to right. If the initial sequence of tokens exactly matches a previously processed prefix byte-for-byte, the inference cluster loads the precomputed KV tensors directly from GPU High Bandwidth Memory (HBM).
However, traditional MCP client implementations routinely break prefix cache reuse due to two subtle anti-patterns:
- ▸Non-deterministic tool enumeration: When multiple MCP servers launch concurrently over stdio or SSE transports, their responses arrive in arbitrary network order. If tool schemas are appended in arrival order, the token sequence changes between turns, invalidating the entire GPU KV cache.
- ▸Cosmetic schema bloat: Standard JSON Schema definitions generated by tool frameworks (such as Pydantic, Zod, or JSON Schema Draft 7) include extensive cosmetic metadata properties (
title,$schema,examples,default, and deep nesting) that add zero semantic value to model reasoning while consuming tens of thousands of tokens.
By stabilizing the prefix order and stripping non-essential schema tokens before invoking LLM inference, the host maximizes KV-cache hits across all conversation turns.
2. Prerequisites and Environment Matrix
Before implementing prefix caching and compression middleware, verify your local development environment satisfies the runtime requirements:
| Component | Minimum Version | Recommended | Notes |
|---|---|---|---|
| Node.js | >= 20.0.0 LTS | v22.12.0 LTS | Required for ESM stream processing |
| Python | >= 3.11.0 | 3.12.4 | Required for fast dictionary filtering |
| @modelcontextprotocol/sdk | 1.6.0 | 1.7.0 | Core TypeScript MCP protocol library |
| Anthropic SDK (Python) | 0.40.0 | 0.45.0 | Required for cache_control ephemeral syntax |
| OpenAI SDK (Python) | 1.50.0 | 1.58.0 | Automatic prefix caching enabled by default |
Ensure your target model supports prompt caching:
- ▸Anthropic:
claude-3-7-sonnet-20250219,claude-3-5-sonnet-20241022, andclaude-3-5-haiku-20241022(minimum cacheable prompt length is 1,024 tokens for Sonnet and 2,048 tokens for Haiku). - ▸OpenAI:
gpt-4o,gpt-4o-mini, ando1(automatically caches prompts exceeding 1,024 tokens). - ▸Google:
gemini-2.5-flashandgemini-2.5-pro(explicit context caching available via Google GenAI SDK).
3. Multi-Host Configuration Snippets
To ensure your desktop and CLI clients leverage caching and preserve predictable tool schemas, configure host environments with static definitions and pinned transports.
Claude Desktop Configuration
Claude Desktop automatically caches the system prompt and tool definitions when the combined prefix exceeds the model's threshold. Place pinned servers in your operating system's configuration file:
{
"mcpServers": {
"filesystem": {
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-filesystem", "/workspace"]
},
"postgres": {
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-postgres", "postgresql:///analytics"]
},
"github": {
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-github"],
"env": {
"GITHUB_PERSONAL_ACCESS_TOKEN": "ghp_EXAMPLE_TOKEN_FOR_AUTHENTICATION"
}
}
}
}Cursor IDE Configuration (.cursor/mcp.json)
When working inside Cursor, declare MCP servers with explicit environment isolation to prevent dynamic token drift:
{
"mcpServers": {
"filesystem": {
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-filesystem", "./src"]
},
"postgres": {
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-postgres", "postgresql:///analytics"]
}
}
}OpenAI Codex CLI Configuration (~/.codex/config.toml)
For developers leveraging the OpenAI Codex CLI, register MCP tool providers with consistent name flags:
[mcp.servers.filesystem]
command = "npx"
args = ["-y", "@modelcontextprotocol/server-filesystem", "/workspace"]
[mcp.servers.postgres]
command = "npx"
args = ["-y", "@modelcontextprotocol/server-postgres", "postgresql:///analytics"]4. Step-by-Step Tool Discovery, Compression, and Caching
Step 1: Discover Tools via JSON-RPC 2.0
When the agent boots, it sends a tools/list request to each running MCP server:
{
"jsonrpc": "2.0",
"id": 1,
"method": "tools/list",
"params": {}
}The server responds with comprehensive schema descriptions that include unnecessary verbose metadata:
{
"jsonrpc": "2.0",
"id": 1,
"result": {
"tools": [
{
"name": "query_database",
"description": "Execute arbitrary read-only SQL queries against the analytical PostgreSQL cluster and return row records in structured format.",
"inputSchema": {
"$schema": "https://json-schema.org/draft-07/schema#",
"title": "QueryDatabaseParameters",
"type": "object",
"properties": {
"sql": {
"type": "string",
"title": "SQL Query",
"description": "A valid SQL SELECT statement targeting public reporting views.",
"examples": ["SELECT id, email FROM users LIMIT 10;"]
},
"timeout_ms": {
"type": "integer",
"title": "Query Timeout",
"description": "Maximum execution time in milliseconds before query cancellation.",
"default": 5000
}
},
"required": ["sql"]
}
}
]
}
}Step 2: AST Schema Compression Algorithm
To eliminate token waste, run every returned tool schema through an AST cleaner that strips cosmetic JSON Schema keys while preserving parameter validation boundaries:
// mcp-schema-compressor.ts
export interface McpRawTool {
name: string;
description?: string;
inputSchema?: Record<string, any>;
}
export interface CompressedTool {
name: string;
description: string;
input_schema: Record<string, any>;
}
const DISCARD_KEYS = new Set([
"$schema",
"title",
"examples",
"default",
"additionalProperties"
]);
function cleanSchemaNode(node: any): any {
if (!node || typeof node !== "object") {
return node;
}
if (Array.isArray(node)) {
return node.map(cleanSchemaNode);
}
const cleaned: Record<string, any> = {};
for (const [key, value] of Object.entries(node)) {
if (DISCARD_KEYS.has(key)) {
continue;
}
// Truncate overly verbose property descriptions to 80 characters
if (key === "description" && typeof value === "string" && value.length > 80) {
cleaned[key] = value.substring(0, 77) + "...";
continue;
}
cleaned[key] = cleanSchemaNode(value);
}
return cleaned;
}
export function compressAndSortTools(tools: McpRawTool[]): CompressedTool[] {
// 1. Sort tools deterministically by name to ensure stable token sequences
const sorted = [...tools].sort((a, b) => a.name.localeCompare(b.name));
// 2. Compress descriptions and strip JSON Schema metadata bloat
return sorted.map((tool) => ({
name: tool.name,
description: (tool.description || "").substring(0, 140),
input_schema: cleanSchemaNode(tool.inputSchema || { type: "object", properties: {} })
}));
}Step 3: Injecting Anthropic Ephemeral Cache Breakpoints
When calling the Anthropic Messages API, mark the final tool in the deterministically sorted array with the cache_control parameter:
# cached_mcp_orchestrator.py
import anthropic
from typing import List, Dict, Any
def build_cached_tool_payload(compressed_tools: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
"""
Format compressed tools and place an ephemeral cache breakpoint on the
final tool in the sorted catalog.
"""
payload = []
total_tools = len(compressed_tools)
for idx, tool in enumerate(compressed_tools):
entry = {
"name": tool["name"],
"description": tool["description"],
"input_schema": tool["input_schema"]
}
# Inject cache breakpoint at the boundary of static tool declarations
if idx == total_tools - 1:
entry["cache_control"] = {"type": "ephemeral"}
payload.append(entry)
return payload
client = anthropic.Anthropic()
def execute_cached_turn(conversation_history: List[Dict[str, Any]], tools: List[Dict[str, Any]]):
formatted_tools = build_cached_tool_payload(tools)
response = client.messages.create(
model="claude-3-7-sonnet-20250219",
max_tokens=2048,
tools=formatted_tools,
messages=conversation_history
)
usage = response.usage
cached_read = getattr(usage, "cache_read_input_tokens", 0)
cached_write = getattr(usage, "cache_creation_input_tokens", 0)
print(f"[Cache Metrics] Write Tokens: {cached_write} | Read Hit Tokens: {cached_read}")
return responseOn Turn 1, cache_creation_input_tokens reports the size of the combined system prompt and compressed tool declarations. On Turn 2 through Turn 20, cache_read_input_tokens captures the full prefix size while billed at a ninety percent discount, yielding near-instantaneous first-token responses.
5. Common Errors and Production Troubleshooting
Error 1: 0% Cache Read Hits Despite Identical Tools
- ▸Symptom:
cache_read_input_tokensremains zero across all conversation turns, incurring full pricing on every request. - ▸Root Cause: Tool schemas are aggregated dynamically from asynchronous background processes without sorting. If Server A finishes before Server B on Turn 1, but Server B finishes before Server A on Turn 2, the serialized JSON prompt prefix differs, forcing the foundation model to discard the cached KV tensors.
- ▸Resolution: Always pipe tool arrays through a strict lexical sort function
tools.sort((a, b) => a.name.localeCompare(b.name))before generating API request frames.
Error 2: 400 invalid_request_error: cache_control limit exceeded
- ▸Symptom: Anthropic API returns HTTP status code 400 with message stating that maximum cache breakpoints have been exceeded.
- ▸Root Cause: The Anthropic API restricts each request to a maximum of four
cache_controlbreakpoints. Developers mistakenly applycache_control: {"type": "ephemeral"}to every tool definition in a loop rather than strictly to the final tool in the array. - ▸Resolution: Ensure your code checks
if idx == len(tools) - 1:so exactly one breakpoint marks the entire tool list prefix.
Error 3: Windows CRLF Invalidation in Cross-Platform Environments
- ▸Symptom: Cache hits function reliably on Linux CI/CD runners but consistently drop to zero when running locally on Windows workstations.
- ▸Root Cause: System prompts and JSON serialization libraries on Windows format multiline string descriptions with
\r\nline breaks, whereas Linux and macOS use\n. Because tokenizers assign distinct integer token IDs to carriage returns, cross-platform cache states fail to align. - ▸Resolution: Normalize all prompt strings and serialized schemas using
.replace(/\r\n/g, '\n')prior to dispatching HTTP or stdio requests.
6. External Citations and Security Guardrails
Official Specifications and References
- ▸Anthropic Prompt Caching Guide (opens in a new tab) — Official developer documentation on ephemeral cache breakpoints, 5-minute TTL policies, and usage metrics.
- ▸OpenAI Prompt Caching Documentation (opens in a new tab) — Architectural reference for automated prefix caching across GPT-4o and o1 model endpoints.
- ▸Model Context Protocol Specification (opens in a new tab) — Official JSON-RPC 2.0 schema specification for
tools/listand client-host discovery lifecycles.
Security and Isolation Guardrails
When engineering cached tool prefixes, enforce strict tenant boundary controls:
- ▸Never cache tenant-specific identifiers in global prefixes: If multiple user sessions share a centralized agent gateway, do not include organization IDs, personal access tokens, or user names in the cached tool description block. Dynamic context belongs in user turns after the cache boundary.
- ▸Sanitize dynamic tool parameters: Tools that inject runtime metadata into schema titles or descriptions invalidate caching across instances. Keep tool definitions static and stateless.
Build your full agent toolstack in the Visual Generator
Combine verified MCP servers with databases, search APIs, and memory graphs in a single configuration file.
Did this setup guide work with your AI host?
Help test this configuration across Claude, Cursor, and Codex CLI. Be the first to report:
Written by Rad Tome
Lead AI Systems Architect & Maintainer, MCP Codex
Specializing in Model Context Protocol (MCP) integrations, autonomous AI agent orchestration, and distributed developer toolchains. Researches and benchmarks production MCP client-server architectures across OpenAI Codex, Claude, and Cursor.
Related Guides
Multi-Agent Orchestration with MCP Swarms
Architect autonomous multi-agent swarms using Model Context Protocol. Implement hierarchical delegation, state synchronization, and consensus protocols.
AI FrameworksAutonomous Goal Loops for AI Agents with MCP
Architecting autonomous /goal execution loops with Model Context Protocol. Learn hierarchical task decomposition, state checkpointing, and convergence.
AI FrameworksGEM Framework: Agentic RL and MCP Environments
Integrate the General Experience Maker (GEM) framework with Model Context Protocol (MCP) servers to train and benchmark agentic LLMs via reinforcement learning.