Rate Limiting and Retry Strategies with MCP (Model Context Protocol): Complete Guide

Rate Limiting and Retry Strategies with MCP (Model Context Protocol): Complete Guide

The Model Context Protocol (MCP) has emerged as a powerful standard for connecting AI assistants to external data sources, tools, and services. However, as with any networked protocol that interacts with APIs, databases, and third-party services, you will inevitably encounter rate limits, transient failures, and network hiccups. Building robust MCP servers and clients requires a thoughtful approach to rate limiting and retry strategies. This guide walks you through everything you need to know to make your MCP integrations resilient, predictable, and production-ready.

What Is Rate Limiting in the Context of MCP?

Rate limiting is the practice of controlling the rate at which requests are sent or received. In the MCP ecosystem, rate limiting applies on two fronts. First, your MCP server may call upstream APIs (such as GitHub, Slack, or a database) that impose their own rate limits. Second, your MCP server itself may need to limit how many requests it accepts from clients to protect resources and ensure fair usage.

The Model Context Protocol defines a JSON-RPC based communication layer between clients (like Claude Desktop or custom AI applications) and servers that expose tools, resources, and prompts. Because MCP servers often act as bridges to external services, they inherit all the rate limiting concerns of those services. If your MCP tool calls the OpenAI API, the GitHub API, and a PostgreSQL database, you need to manage rate limits for each of those independently while also presenting a smooth experience to the MCP client.

Why Rate Limiting and Retries Matter

• Upstream API protection: Most cloud APIs enforce rate limits (e.g., 60 requests per minute). Exceeding them results in HTTP 429 responses that break your tool execution.
• Cost control: LLM-backed tools and paid APIs can generate significant costs if called without restraint.
• Fairness: If your MCP server is shared across multiple users or sessions, one heavy user should not starve others.
• Transient failures: Network timeouts, DNS blips, and temporary 503 errors are common. Retries with backoff dramatically improve reliability.
• User experience: An AI assistant that silently fails because of a rate limit is frustrating. Graceful handling with informative error messages is far better.

Understanding MCP Error Handling

MCP uses JSON-RPC 2.0, which defines a standard error object with a code, message, and optional data field. When your MCP tool encounters a rate limit from an upstream service, you should translate that into a meaningful MCP error rather than letting the request fail opaquely. The MCP specification defines several standard error codes, including -32001 for request cancelled and the JSON-RPC standard range for server errors. For rate limiting, you can use custom error codes in the -32000 to -32099 range reserved for implementation-defined server errors.

Implementing a Token Bucket Rate Limiter

The token bucket algorithm is one of the most popular rate limiting strategies because it allows bursts while maintaining an average rate. Tokens are added to a bucket at a fixed rate, and each request consumes a token. If the bucket is empty, the request is rejected or delayed. Here is a TypeScript implementation you can use inside an MCP server:

class TokenBucket {
  private tokens: number;
  private lastRefill: number;
  private readonly capacity: number;
  private readonly refillRate: number; // tokens per second

  constructor(capacity: number, refillRate: number) {
    this.capacity = capacity;
    this.refillRate = refillRate;
    this.tokens = capacity;
    this.lastRefill = Date.now();
  }

  private refill(): void {
    const now = Date.now();
    const elapsed = (now - this.lastRefill) / 1000;
    this.tokens = Math.min(this.capacity, this.tokens + elapsed * this.refillRate);
    this.lastRefill = now;
  }

  tryConsume(count: number = 1): boolean {
    this.refill();
    if (this.tokens >= count) {
      this.tokens -= count;
      return true;
    }
    return false;
  }

  getAvailableTokens(): number {
    this.refill();
    return this.tokens;
  }
}

You can instantiate one bucket per upstream service or per user session. For example, if your MCP server exposes a GitHub tool, create a bucket sized to match GitHub's rate limit for your token.

Building an MCP Server with Rate Limiting

Let us build a complete MCP server that exposes a tool for fetching repository information from GitHub, with built-in rate limiting. We will use the official MCP TypeScript SDK:

import { Server } from "@modelcontextprotocol/sdk/server/index.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
import {
  CallToolRequestSchema,
  ListToolsRequestSchema,
} from "@modelcontextprotocol/sdk/types.js";

const githubBucket = new TokenBucket(30, 0.5); // 30 burst, 0.5/sec

const server = new Server(
  { name: "github-mcp-server", version: "1.0.0" },
  { capabilities: { tools: {} } }
);

server.setRequestHandler(ListToolsRequestSchema, async () => ({
  tools: [
    {
      name: "get_repo",
      description: "Fetch repository metadata from GitHub",
      inputSchema: {
        type: "object",
        properties: {
          owner: { type: "string" },
          repo: { type: "string" },
        },
        required: ["owner", "repo"],
      },
    },
  ],
}));

server.setRequestHandler(CallToolRequestSchema, async (request) => {
  if (request.params.name !== "get_repo") {
    throw new Error(`Unknown tool: ${request.params.name}`);
  }

  if (!githubBucket.tryConsume()) {
    return {
      content: [
        {
          type: "text",
          text: "Rate limit exceeded for GitHub API. Please try again in a few seconds.",
        },
      ],
      isError: true,
    };
  }

  const { owner, repo } = request.params.arguments as {
    owner: string;
    repo: string;
  };

  const response = await fetch(
    `https://api.github.com/repos/${owner}/${repo}`,
    {
      headers: {
        Accept: "application/vnd.github.v3+json",
        Authorization: `Bearer ${process.env.GITHUB_TOKEN}`,
      },
    }
  );

  if (response.status === 404) {
    return {
      content: [{ type: "text", text: `Repository ${owner}/${repo} not found.` }],
      isError: true,
    };
  }

  if (!response.ok) {
    return {
      content: [
        {
          type: "text",
          text: `GitHub API error: ${response.status} ${response.statusText}`,
        },
      ],
      isError: true,
    };
  }

  const data = await response.json();
  return {
    content: [
      {
        type: "text",
        text: JSON.stringify(
          {
            name: data.name,
            description: data.description,
            stars: data.stargazers_count,
            forks: data.forks_count,
            open_issues: data.open_issues_count,
          },
          null,
          2
        ),
      },
    ],
  };
});

const transport = new StdioServerTransport();
await server.connect(transport);

Notice how the rate limit check happens before the HTTP call. This prevents wasting an upstream API call when we already know we are over budget. The error is returned as MCP content with isError: true, which tells the client that the tool execution failed but in a controlled way.

Retry Strategies: Exponential Backoff with Jitter

When a request fails due to a transient error or a 429 response, retrying immediately is usually counterproductive. If the server is overloaded, hammering it again will only make things worse. Exponential backoff increases the wait time between retries, giving the upstream service time to recover. Adding jitter (randomized delay) prevents the thundering herd problem where many clients retry at the same moment.

async function fetchWithRetry(
  url: string,
  options: RequestInit,
  maxRetries: number = 5,
  baseDelayMs: number = 1000
): Promise<Response> {
  let lastError: Error | null = null;

  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    try {
      const response = await fetch(url, options);

      if (response.status === 429 || response.status >= 500) {
        if (attempt === maxRetries) {
          return response; // give up, return the error response
        }

        // Check for Retry-After header
        const retryAfter = response.headers.get("Retry-After");
        let delayMs: number;

        if (retryAfter) {
          delayMs = parseInt(retryAfter, 10) * 1000;
        } else {
          // Exponential backoff with full jitter
          const exponentialDelay = baseDelayMs * Math.pow(2, attempt);
          delayMs = Math.random() * exponentialDelay;
        }

        await new Promise((resolve) => setTimeout(resolve, delayMs));
        continue;
      }

      return response;
    } catch (error) {
      lastError = error as Error;
      if (attempt === maxRetries) {
        throw error;
      }

      const exponentialDelay = baseDelayMs * Math.pow(2, attempt);
      const delayMs = Math.random() * exponentialDelay;
      await new Promise((resolve) => setTimeout(resolve, delayMs));
    }
  }

  throw lastError ?? new Error("Max retries exceeded");
}

This function handles both HTTP-level errors (429, 5xx) and network-level errors (timeouts, connection resets). It respects the Retry-After header when present, which is the polite way for servers to tell clients exactly how long to wait. When no header is provided, it falls back to exponential backoff with full jitter.

Integrating Retries into MCP Tool Handlers

Now let us update our MCP server to use the retry-enabled fetch function. We will also add a circuit breaker pattern to stop calling an upstream service that is consistently failing:

class CircuitBreaker {
  private failures = 0;
  private lastFailureTime = 0;
  private state: "closed" | "open" | "half-open" = "closed";

  constructor(
    private readonly threshold: number = 5,
    private readonly resetTimeoutMs: number = 60000
  ) {}

  async execute<T>(fn: () => Promise<T>): Promise<T> {
    if (this.state === "open") {
      if (Date.now() - this.lastFailureTime > this.resetTimeoutMs) {
        this.state = "half-open";
      } else {
        throw new Error("Circuit breaker is open. Service temporarily unavailable.");
      }
    }

    try {
      const result = await fn();
      this.failures = 0;
      this.state = "closed";
      return result;
    } catch (error) {
      this.failures++;
      this.lastFailureTime = Date.now();

      if (this.failures >= this.threshold) {
        this.state = "open";
      }

      throw error;
    }
  }
}

const githubBreaker = new CircuitBreaker(5, 60000);

server.setRequestHandler(CallToolRequestSchema, async (request) => {
  if (request.params.name !== "get_repo") {
    throw new Error(`Unknown tool: ${request.params.name}`);
  }

  if (!githubBucket.tryConsume()) {
    return {
      content: [
        {
          type: "text",
          text: "Rate limit exceeded. Please retry shortly.",
        },
      ],
      isError: true,
    };
  }

  const { owner, repo } = request.params.arguments as {
    owner: string;
    repo: string;
  };

  try {
    const response = await githubBreaker.execute(() =>
      fetchWithRetry(
        `https://api.github.com/repos/${owner}/${repo}`,
        {
          headers: {
            Accept: "application/vnd.github.v3+json",
            Authorization: `Bearer ${process.env.GITHUB_TOKEN}`,
          },
        }
      )
    );

    if (!response.ok) {
      return {
        content: [
          {
            type: "text",
            text: `GitHub API error after retries: ${response.status}`,
          },
        ],
        isError: true,
      };
    }

    const data = await response.json();
    return {
      content: [
        {
          type: "text",
          text: JSON.stringify(
            {
              name: data.name,
              stars: data.stargazers_count,
              description: data.description,
            },
            null,
            2
          ),
        },
      ],
    };
  } catch (error) {
    return {
      content: [
        {
          type: "text",
          text: `Failed to fetch repository: ${(error as Error).message}`,
        },
      ],
      isError: true,
    };
  }
});

Per-User and Per-Session Rate Limiting

In a multi-user MCP deployment, a single global rate limiter is insufficient. One user could exhaust the budget, blocking everyone else. The solution is to maintain separate rate limiters keyed by user or session identifier. MCP clients can include metadata that identifies the session, and you can use that to partition your rate limiting:

class RateLimiterManager {
  private limiters = new Map<string, TokenBucket>();

  getLimiter(key: string): TokenBucket {
    let limiter = this.limiters.get(key);
    if (!limiter) {
      limiter = new TokenBucket(10, 1); // 10 burst, 1/sec per user
      this.limiters.set(key, limiter);
    }
    return limiter;
  }

  tryConsume(key: string, count: number = 1): boolean {
    return this.getLimiter(key).tryConsume(count);
  }

  // Clean up stale limiters periodically
  cleanup(maxAgeMs: number = 3600000): void {
    // In production, track last access time and remove old entries
    // This is a placeholder for the cleanup logic
  }
}

const limiterManager = new RateLimiterManager();

// In your tool handler, extract a user identifier from the request
// context or generate one based on the transport session
function getSessionKey(request: any): string {
  // MCP does not yet standardize user identity, so you may need
  // to rely on transport-level session IDs or custom headers
  return request._meta?.sessionId ?? "anonymous";
}

Client-Side Retry Strategies for MCP Clients

Rate limiting is not only the server's responsibility. If you are building an MCP client (for example, an AI application that connects to multiple MCP servers), you should also implement retry logic on the client side. When an MCP server returns an error, the client can decide whether to retry the tool call, ask the LLM to try a different approach, or surface the error to the user:

import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";

async function callToolWithRetry(
  client: Client,
  toolName: string,
  args: Record<string, unknown>,
  maxRetries: number = 3
): Promise<any> {
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    try {
      const result = await client.callTool({
        name: toolName,
        arguments: args,
      });

      if (result.isError) {
        const text = result.content?.[0]?.text ?? "";
        if (text.includes("Rate limit") && attempt < maxRetries) {
          const delayMs = 2000 * Math.pow(2, attempt) + Math.random() * 500;
          await new Promise((r) => setTimeout(r, delayMs));
          continue;
        }
        return result;
      }

      return result;
    } catch (error) {
      if (attempt === maxRetries) {
        throw error;
      }
      const delayMs = 1000 * Math.pow(2, attempt) + Math.random() * 500;
      await new Promise((r) => setTimeout(r, delayMs));
    }
  }
}

const transport = new StdioClientTransport({
  command: "node",
  args: ["github-mcp-server.js"],
});

const client = new Client(
  { name: "my-ai-app", version: "1.0.0" },
  { capabilities: {} }
);

await client.connect(transport);

const result = await callToolWithRetry(client, "get_repo", {
  owner: "modelcontextprotocol",
  repo: "python-sdk",
});

console.log(result);

Best Practices for Rate Limiting and Retries in MCP

• Always respect Retry-After headers: When an upstream API tells you how long to wait, honor it. Ignoring this header and retrying on your own schedule can get your API key banned.
• Use jitter in all retry logic: Synchronized retries from multiple clients or sessions can create load spikes. Full jitter (randomizing between 0 and the exponential delay) is generally the safest choice.
• Set a maximum retry count: Infinite retries can hang your MCP server and frustrate users. Five retries is a reasonable default for most use cases.
• Return meaningful errors to the LLM: When a tool fails, include enough context in the error text that the AI model can understand what happened and decide whether to try a different approach.
• Log rate limit events: Track when rate limits are hit so you can identify which tools or users are the heaviest consumers and adjust your limits accordingly.
• Separate rate limits per upstream service: Do not share a single rate limiter across multiple APIs. Each upstream has its own limits, and mixing them will cause unnecessary throttling.
• Consider request prioritization: If some tool calls are more important than others (for example, a read vs. a bulk write), you may want priority queues rather than a simple first-come-first-served limiter.
• Test under load: Simulate rate limit scenarios in your test suite by mocking upstream 429 responses. This ensures your retry and error handling logic works correctly before production.
• Use idempotency keys for write operations: If a tool call mutates data and you retry it, you risk duplicate writes. Include an idempotency key so the upstream service can deduplicate safely.

Monitoring and Observability

Rate limiting and retry logic are only as good as your ability to observe their behavior. You should log every rate limit rejection, every retry attempt, and every circuit breaker state transition. Here is a simple logging wrapper you can integrate:

class ObservableTokenBucket extends TokenBucket {
  constructor(
    capacity: number,
    refillRate: number,
    private readonly label: string
  ) {
    super(capacity, refillRate);
  }

  tryConsume(count: number = 1): boolean {
    const allowed = super.tryConsume(count);
    if (!allowed) {
      console.warn(
        JSON.stringify({
          event: "rate_limit_exceeded",
          label: this.label,
          timestamp: new Date().toISOString(),
        })
      );
    }
    return allowed;
  }
}

function logRetry(attempt: number, maxRetries: number, error: string): void {
  console.info(
    JSON.stringify({
      event: "retry_attempt",
      attempt,
      maxRetries,
      error,
      timestamp: new Date().toISOString(),
    })
  );
}

In a production environment, forward these logs to your observability platform (Datadog, Grafana, CloudWatch, etc.) and set up alerts for spikes in rate limit rejections or circuit breaker openings.

Conclusion

Rate limiting and retry strategies are essential components of any production-grade MCP server or client. By implementing token bucket rate limiters, exponential backoff with jitter, circuit breakers, and per-user isolation, you can build MCP integrations that gracefully handle the realities of upstream API constraints and transient network failures. The key is to be proactive rather than reactive: design your error handling before you hit production traffic, test it under simulated failure conditions, and monitor it continuously once deployed. With the patterns and code examples in this guide, you have everything you need to make your MCP-powered applications resilient, fair, and reliable.