Streaming Responses in Production with MCP (Model Context Protocol): Complete Guide
The Model Context Protocol (MCP) has emerged as a powerful standard for connecting AI models to external tools, resources, and data sources. As applications scale beyond simple request-response patterns, streaming responses become essential for delivering responsive, real-time experiences to end users. This guide walks through everything you need to know about implementing streaming responses with MCP in production environments.
What Is Streaming in MCP?
Streaming in MCP refers to the ability to send and receive data incrementally over a persistent connection rather than waiting for a complete response before returning anything to the client. The MCP specification supports streaming through its JSON-RPC 2.0 transport layer, which allows servers to push notifications and partial results back to clients as they become available.
In a traditional request-response model, a client sends a request and blocks until the server completes processing. With streaming, the server can emit intermediate progress updates, partial tool outputs, or token-by-token model responses. This is particularly valuable when a tool invocation involves long-running operations such as web scraping, database queries across large datasets, or chained LLM calls.
Why Streaming Matters in Production
- Perceived performance: Users see progress immediately rather than staring at a spinner for 30 seconds.
- Timeout avoidance: Long-running operations can keep connections alive with heartbeat updates, preventing gateway timeouts.
- Token-by-token delivery: For LLM-backed tools, streaming tokens reduces time-to-first-token dramatically.
- Better cancellation: Clients can abort operations mid-stream when they detect the output is no longer needed.
- Resource efficiency: Servers can begin releasing memory as chunks are sent rather than buffering entire responses.
Understanding the MCP Transport Layer
MCP supports two primary transports: stdio and HTTP with Server-Sent Events (SSE). For streaming in production, the HTTP+SSE transport is the standard choice because it provides a persistent, unidirectional stream from server to client while allowing the client to send requests over a separate HTTP POST channel.
The newer MCP specification (2025-03-26 and later) also introduces the Streamable HTTP transport, which simplifies the architecture by allowing a single endpoint to handle both directions. We will focus on the Streamable HTTP approach as it is the recommended path for new production deployments.
Setting Up a Streaming MCP Server
Let us build a streaming MCP server using the official TypeScript SDK. The server will expose a tool that performs a simulated long-running search operation and streams progress updates back to the client.
First, install the required dependencies:
npm install @modelcontextprotocol/sdk express
npm install -D typescript @types/express @types/node tsx
Next, create the server file. This example uses the Streamable HTTP transport and registers a tool that yields progress notifications as it works:
// server.ts
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/streamableHttp.js";
import express from "express";
import { z } from "zod";
const app = express();
app.use(express.json());
const server = new McpServer({
name: "streaming-search-server",
version: "1.0.0",
});
server.registerTool(
"search_documents",
{
title: "Search Documents",
description: "Searches documents and streams progress updates.",
inputSchema: {
query: z.string().describe("The search query"),
maxResults: z.number().optional().default(10),
},
},
async (args, { meta }) => {
const { query, maxResults } = args;
const results: string[] = [];
for (let i = 0; i < maxResults; i++) {
// Simulate async work per result
await new Promise((resolve) => setTimeout(resolve, 500));
const result = `Result ${i + 1} for "${query}"`;
results.push(result);
// Send a progress notification to the client
server.server.notification({
method: "notifications/progress",
params: {
progress: ((i + 1) / maxResults) * 100,
total: maxResults,
message: `Found ${i + 1} of ${maxResults} results`,
},
});
}
return {
content: [
{
type: "text",
text: JSON.stringify(results, null, 2),
},
],
};
}
);
// Mount the Streamable HTTP transport
app.post("/mcp", async (req, res) => {
const transport = new StreamableHTTPServerTransport({ sessionIdGenerator: undefined });
transport.onclose = () => {
console.log("Transport closed");
};
await server.connect(transport);
await transport.handleRequest(req, res);
});
app.listen(3000, () => {
console.log("Streaming MCP server running on http://localhost:3000/mcp");
});
The key pattern here is calling server.server.notification() inside the tool handler. This pushes a notification frame through the open SSE stream to the client without closing the response. The final return statement completes the tool call with the full result.
Building a Streaming MCP Client
On the client side, you need to connect to the server, invoke the tool, and listen for incoming notifications. Here is a complete client implementation:
// client.ts
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js";
async function main() {
const transport = new StreamableHTTPClientTransport(
new URL("http://localhost:3000/mcp")
);
const client = new Client({
name: "streaming-client",
version: "1.0.0",
});
// Register a handler for progress notifications
client.setNotificationHandler(
{ method: "notifications/progress" },
(notification) => {
const params = notification.params as {
progress: number;
total: number;
message: string;
};
console.log(
`[Progress] ${params.progress.toFixed(0)}% - ${params.message}`
);
}
);
await client.connect(transport);
console.log("Connected to MCP server");
const result = await client.callTool({
name: "search_documents",
arguments: {
query: "machine learning",
maxResults: 5,
},
});
console.log("\nFinal results:");
for (const content of result.content) {
if (content.type === "text") {
console.log(content.text);
}
}
await client.close();
}
main().catch(console.error);
When you run both the server and client, you will see progress updates printed incrementally as the server processes each result, followed by the final payload. This is the core of streaming with MCP.
Streaming Token-by-Token from an LLM Tool
One of the most common production use cases is wrapping an LLM call inside an MCP tool and streaming tokens back to the client. Here is how to adapt the server to stream tokens from an OpenAI-compatible API:
server.registerTool(
"generate_summary",
{
title: "Generate Summary",
description: "Generates a streaming summary of the provided text.",
inputSchema: {
text: z.string().describe("The text to summarize"),
},
},
async (args) => {
const response = await fetch("https://api.openai.com/v1/chat/completions", {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
},
body: JSON.stringify({
model: "gpt-4o-mini",
stream: true,
messages: [
{
role: "system",
content: "Summarize the following text concisely.",
},
{ role: "user", content: args.text },
],
}),
});
if (!response.body) {
throw new Error("No response body from LLM");
}
const reader = response.body.getReader();
const decoder = new TextDecoder();
let fullText = "";
let buffer = "";
while (true) {
const { done, value } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
const lines = buffer.split("\n");
buffer = lines.pop() || "";
for (const line of lines) {
const trimmed = line.trim();
if (!trimmed.startsWith("data: ")) continue;
const data = trimmed.slice(6);
if (data === "[DONE]") continue;
try {
const parsed = JSON.parse(data);
const delta = parsed.choices?.[0]?.delta?.content;
if (delta) {
fullText += delta;
// Stream each token chunk to the MCP client
server.server.notification({
method: "notifications/progress",
params: {
progress: 0,
message: delta,
streaming: true,
},
});
}
} catch {
// Ignore malformed chunks
}
}
}
return {
content: [{ type: "text", text: fullText }],
};
}
);
The client can then reconstruct the full response by concatenating the message field from each progress notification where streaming is true, while still receiving the complete text in the final tool result.
Best Practices for Production Streaming
1. Always Set Session Management
In production, you should enable session management so that the server can track state across multiple requests from the same client. Use a stable session ID generator and store session state in a durable store such as Redis:
import { randomUUID } from "crypto";
const transport = new StreamableHTTPServerTransport({
sessionIdGenerator: () => randomUUID(),
});
// Restore sessions on reconnect
transport.onsessioninitialized = (sessionId) => {
console.log(`Session initialized: ${sessionId}`);
};
2. Implement Heartbeats for Long Operations
Load balancers and reverse proxies often terminate idle connections after 30-60 seconds. If your tool operation may run longer, send periodic heartbeat notifications to keep the connection alive:
const heartbeatInterval = setInterval(() => {
server.server.notification({
method: "notifications/heartbeat",
params: { timestamp: Date.now() },
});
}, 15000);
// Clear the interval when the operation completes
clearInterval(heartbeatInterval);
3. Handle Backpressure Correctly
Streaming large amounts of data can overwhelm slow clients. Monitor the writable stream state and pause generation when the client cannot keep up. In Node.js, check the writable.writableNeedDrain property and await a drain event before sending more data.
4. Provide Meaningful Error Notifications
Do not wait until the end to report errors. If a streaming operation fails midway, send an error notification immediately so the client can react:
try {
await performLongOperation();
} catch (error) {
server.server.notification({
method: "notifications/error",
params: {
code: "OPERATION_FAILED",
message: error instanceof Error ? error.message : "Unknown error",
},
});
throw error; // Still throw so the tool call returns an error result
}
5. Use Structured Progress Data
Avoid ad-hoc notification schemas. Define a consistent progress schema across all your tools so clients can render progress bars, percentage indicators, and partial content uniformly:
{
"method": "notifications/progress",
"params": {
"progress": 45,
"total": 100,
"stage": "fetching",
"message": "Retrieving page 9 of 20",
"partialResult": null,
"timestamp": 1711324800000
}
}
6. Secure the Streaming Endpoint
Streaming endpoints are particularly vulnerable to abuse because connections stay open longer. Implement the following safeguards:
- Require authentication on the
/mcpendpoint using bearer tokens or session cookies. - Enforce rate limits on connection count per client to prevent resource exhaustion.
- Set a maximum stream duration (for example, 5 minutes) and close connections that exceed it.
- Validate input schemas strictly using Zod or similar libraries before processing begins.
- Log all tool invocations with correlation IDs for observability and auditing.
7. Test Streaming Behavior Explicitly
Streaming bugs are often invisible in unit tests because the final result may still be correct even if notifications are dropped. Write integration tests that assert on the sequence and timing of notifications:
import { describe, it, expect } from "vitest";
describe("search_documents streaming", () => {
it("should emit progress notifications in order", async () => {
const notifications: number[] = [];
client.setNotificationHandler(
{ method: "notifications/progress" },
(notification) => {
notifications.push((notification.params as any).progress);
}
);
await client.callTool({
name: "search_documents",
arguments: { query: "test", maxResults: 3 },
});
expect(notifications).toEqual([33.33, 66.67, 100]);
});
});
Deployment Considerations
When deploying a streaming MCP server behind infrastructure, keep these points in mind. First, disable response buffering at every layer. Nginx, for example, buffers responses by default; you must set proxy_buffering off for the MCP route. Second, ensure your load balancer supports long-lived HTTP connections and does not impose aggressive idle timeouts. AWS Application Load Balancer defaults to 60 seconds, which is often too short; consider using a Network Load Balancer or increasing the idle timeout. Third, run multiple server instances behind a load balancer and use sticky sessions or the MCP session ID to route requests from the same client to the same server instance, since session state is stored in memory by default.
Conclusion
Streaming responses transform MCP from a simple tool-calling protocol into a foundation for real-time, interactive AI applications. By leveraging the Streamable HTTP transport, emitting structured progress notifications, handling backpressure, and following production best practices around security and observability, you can build robust MCP servers that keep users informed at every step of long-running operations. Start with the patterns in this guide, adapt the progress schema to your domain, and always test the streaming behavior itself, not just the final result. With these foundations in place, your MCP-powered tools will deliver the responsive experience that production-grade AI applications demand.