← Back to DevBytes
AI July 19, 2026 17 min read

AI Agent Hosting Compared: Managed vs Self-Hosted vs Serverless

Understanding AI Agent Hosting Models

When you build an AI agent—whether it's a LangChain-based reasoning engine, a custom GPT wrapper with tool access, or a multi-step autonomous workflow—you eventually face a critical decision: where and how should this agent live? The hosting model you choose directly impacts latency, cost, scalability, security, and how easily you can iterate. There are three dominant paradigms: managed platforms, self-hosted infrastructure, and serverless functions. Each carries distinct trade-offs that become magnified when you're dealing with long-running LLM calls, stateful conversations, and tool orchestration.

This tutorial walks through each hosting model in depth, provides complete, runnable code examples for deploying the same AI agent across all three, and lays out best practices so you can make an informed choice for your production workloads.

What Defines an AI Agent Hosting Environment?

Unlike a traditional REST API that responds in milliseconds, an AI agent often needs to sustain open connections for tens of seconds or even minutes. It may chain multiple LLM calls, invoke external APIs, wait for human approval steps, or manage a persistent memory store. The hosting environment must handle:

Your choice of hosting determines how gracefully you can satisfy these requirements. Let's examine each model through the lens of a concrete AI agent: a simple research assistant that takes a user query, performs a web search via a tool, summarizes the results, and returns the answer. We'll deploy this exact agent three different ways.

The Reference AI Agent

Before we compare hosting, here is the agent we'll be deploying. It uses OpenAI's SDK with a custom tool for web search. This code is self-contained and will serve as our baseline. 

# agent_core.py
import os
import json
import asyncio
from openai import OpenAI
import httpx

client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))

# Tool definition
def search_web(query: str) -> str:
    """Search the web for a query and return a summary."""
    # Simulated search - replace with real API like SerpAPI or Tavily
    results = [
        f"Result for '{query}': AI agents are autonomous programs that use LLMs to plan and execute tasks.",
        f"Additional context: Hosting options include managed, self-hosted, and serverless architectures."
    ]
    return "\n".join(results[:2])

TOOLS = [
    {
        "type": "function",
        "function": {
            "name": "search_web",
            "description": "Search the web for information on a given query",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {
                        "type": "string",
                        "description": "The search query"
                    }
                },
                "required": ["query"]
            }
        }
    }
]

async def run_agent(user_query: str) -> str:
    """Run the agent loop: call LLM, execute tools, return final answer."""
    messages = [
        {"role": "system", "content": "You are a helpful research assistant. Use the search_web tool to find information, then synthesize a concise answer."},
        {"role": "user", "content": user_query}
    ]

    # Agent loop - up to 3 tool calls
    for _ in range(3):
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=messages,
            tools=TOOLS,
            tool_choice="auto"
        )
        message = response.choices[0].message

        if message.tool_calls:
            for tool_call in message.tool_calls:
                if tool_call.function.name == "search_web":
                    args = json.loads(tool_call.function.arguments)
                    result = search_web(args["query"])
                    messages.append({
                        "role": "tool",
                        "tool_call_id": tool_call.id,
                        "content": result
                    })
            # Continue loop
            continue
        else:
            # No tool calls - this is the final answer
            return message.content

    return "Agent reached maximum tool call iterations."

# Synchronous wrapper for simpler deployment
def run_agent_sync(user_query: str) -> str:
    return asyncio.run(run_agent(user_query))

This agent is intentionally straightforward—a single query triggers a loop of LLM reasoning and tool execution. In production, you might add memory, multi-step planning, or human-in-the-loop pauses. The hosting implications scale with that complexity.

Model 1: Managed AI Agent Platforms

What It Is

Managed platforms abstract away infrastructure completely. You provide your agent logic (often as a Python function or a configuration file), and the platform handles provisioning, scaling, monitoring, secrets management, and API exposure. Examples include Modal, Replit Agent Hosting, LangChain Cloud, Steamship, and Fly.io Machines (when used with their managed runtime). These platforms typically charge based on compute seconds, with generous free tiers for experimentation.

Why It Matters

Complete Deployment Example (Modal)

Modal lets you decorate Python functions and deploy them as scalable endpoints. The platform keeps your function "warm" and handles incoming HTTP requests. Here's how to deploy our research agent: 

# deploy_managed_modal.py
import modal
from agent_core import run_agent_sync

# Create a Modal app
app = modal.App("research-agent-managed")

# Define a container image with dependencies
image = modal.Image.debian_slim().pip_install(
    "openai",
    "httpx"
)

# Mount the agent code so it's available inside the container
AGENT_CODE = modal.Mount.from_local_file(
    "./agent_core.py",
    remote_path="/root/agent_core.py"
)

@app.function(
    image=image,
    mounts=[AGENT_CODE],
    secrets=[modal.Secret.from_name("openai-api-key")],
    # Allow up to 5 minutes for agent execution
    timeout=300,
    # Keep a container warm for low latency
    container_idle_timeout=600
)
@modal.web_endpoint(method="POST")
def run_research_agent(request: dict):
    """
    Managed endpoint that runs the agent.
    Request body: {"query": "What are AI agents?"}
    """
    query = request.get("query", "")
    if not query:
        return {"error": "Missing 'query' field"}, 400
    
    result = run_agent_sync(query)
    return {"answer": result}

# Deploy with: modal deploy deploy_managed_modal.py
# The CLI will output a public URL like:
# https://your-org--research-agent-managed-run-research-agent.modal.run

With this single file and one CLI command, you get a production-grade HTTPS endpoint. Modal handles TLS termination, request queuing, autoscaling (up to hundreds of containers), and log streaming. The timeout=300 parameter is critical—it tells the platform to allow up to 5 minutes per invocation, which accommodates multiple LLM round-trips. The container_idle_timeout keeps a warm instance ready, avoiding cold starts of 2–5 seconds on subsequent requests.

Best Practices for Managed Hosting

Model 2: Self-Hosted Infrastructure

What It Is

Self-hosting means you run your agent on infrastructure you control: a VPS (DigitalOcean, AWS EC2, Hetzner), a Kubernetes cluster, or even on-premises hardware. You package the agent as a web service (typically FastAPI or Flask), wrap it in a Docker container, and manage the full stack yourself—load balancers, TLS certificates, monitoring, log aggregation, and scaling policies.

Why It Matters

Complete Deployment Example (FastAPI + Docker on a VPS)

We'll build a production-ready FastAPI service with streaming support, wrap it in Docker, and deploy it to a Linux VPS behind Nginx. 

# server.py - FastAPI service for the research agent
import os
import sys
import asyncio
import json
from fastapi import FastAPI, Request, HTTPException
from fastapi.responses import StreamingResponse, JSONResponse
from pydantic import BaseModel
from openai import OpenAI

# Add parent to path for agent_core import
sys.path.insert(0, "/app")
from agent_core import run_agent

app = FastAPI(title="Research Agent API", version="1.0.0")

class QueryRequest(BaseModel):
    query: str
    stream: bool = False

class QueryResponse(BaseModel):
    answer: str
    tool_calls_made: int

@app.on_event("startup")
async def startup_event():
    """Verify API key is present on boot."""
    if not os.environ.get("OPENAI_API_KEY"):
        raise RuntimeError("OPENAI_API_KEY environment variable is required")
    print("Agent server started successfully")

@app.post("/agent/run", response_model=QueryResponse)
async def run_agent_endpoint(request: QueryRequest):
    """Run the agent synchronously and return the result."""
    try:
        result = await run_agent(request.query)
        return QueryResponse(answer=result, tool_calls_made=0)
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

@app.post("/agent/stream")
async def stream_agent_endpoint(request: QueryRequest):
    """Stream the agent's reasoning steps as Server-Sent Events."""
    async def event_generator():
        client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))
        messages = [
            {"role": "system", "content": "You are a helpful research assistant."},
            {"role": "user", "content": request.query}
        ]
        tools = [
            {
                "type": "function",
                "function": {
                    "name": "search_web",
                    "description": "Search the web",
                    "parameters": {
                        "type": "object",
                        "properties": {"query": {"type": "string"}},
                        "required": ["query"]
                    }
                }
            }
        ]
        
        yield f"data: {json.dumps({'event': 'started', 'query': request.query})}\n\n"
        
        for _ in range(3):
            response = client.chat.completions.create(
                model="gpt-4o",
                messages=messages,
                tools=tools,
                tool_choice="auto",
                stream=True
            )
            
            # Collect streaming chunks
            tool_calls_buffer = {}
            content_buffer = ""
            
            for chunk in response:
                delta = chunk.choices[0].delta
                if delta.content:
                    content_buffer += delta.content
                    yield f"data: {json.dumps({'event': 'thought', 'text': delta.content})}\n\n"
                if delta.tool_calls:
                    for tc in delta.tool_calls:
                        idx = tc.index
                        if idx not in tool_calls_buffer:
                            tool_calls_buffer[idx] = {"id": "", "name": "", "arguments": ""}
                        if tc.id:
                            tool_calls_buffer[idx]["id"] = tc.id
                        if tc.function and tc.function.name:
                            tool_calls_buffer[idx]["name"] = tc.function.name
                        if tc.function and tc.function.arguments:
                            tool_calls_buffer[idx]["arguments"] += tc.function.arguments
            
            if tool_calls_buffer:
                # Execute tools
                for tc_data in tool_calls_buffer.values():
                    yield f"data: {json.dumps({'event': 'tool_call', 'name': tc_data['name'], 'args': tc_data['arguments']})}\n\n"
                    # Simulate tool result
                    result = f"Search results for the query..."
                    messages.append({"role": "tool", "tool_call_id": tc_data['id'], "content": result})
                continue
            else:
                yield f"data: {json.dumps({'event': 'done', 'answer': content_buffer})}\n\n"
                break
        
        yield "data: {\"event\": \"finished\"}\n\n"
    
    return StreamingResponse(
        event_generator(),
        media_type="text/event-stream",
        headers={
            "Cache-Control": "no-cache",
            "Connection": "keep-alive",
            "X-Accel-Buffering": "no"  # Disable Nginx buffering for SSE
        }
    )

@app.get("/health")
async def health_check():
    return {"status": "healthy", "uptime": "ok"}

Now the Dockerfile that packages everything:

# Dockerfile
FROM python:3.12-slim

WORKDIR /app

# Install system dependencies
RUN apt-get update && apt-get install -y \
    curl \
    && rm -rf /var/lib/apt/lists/*

# Copy requirements and install
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy application code
COPY agent_core.py .
COPY server.py .

# Create non-root user for security
RUN useradd -m -s /bin/bash agentuser && chown -R agentuser:agentuser /app
USER agentuser

EXPOSE 8080

# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
    CMD curl -f http://localhost:8080/health || exit 1

CMD ["uvicorn", "server:app", "--host", "0.0.0.0", "--port", "8080", "--workers", "2"]

The requirements file:

# requirements.txt
fastapi==0.115.0
uvicorn[standard]==0.30.6
openai==1.55.0
httpx==0.28.1
pydantic==2.10.0

Deployment commands for a VPS running Ubuntu 22.04:

# On your local machine - build and push the image
docker build -t research-agent:v1 .
docker save research-agent:v1 | gzip > research-agent.tar.gz

# Copy to VPS
scp research-agent.tar.gz user@your-vps:/home/user/

# On the VPS - load and run
sudo docker load < research-agent.tar.gz

# Create a .env file with your API key
echo "OPENAI_API_KEY=sk-your-key-here" > /home/user/agent.env

# Run the container with restart policy
sudo docker run -d \
    --name research-agent \
    --env-file /home/user/agent.env \
    --restart unless-stopped \
    -p 127.0.0.1:8080:8080 \
    research-agent:v1

# Set up Nginx as reverse proxy
sudo apt install -y nginx certbot python3-certbot-nginx

# Create Nginx config
sudo tee /etc/nginx/sites-available/research-agent << 'EOF'
server {
    listen 80;
    server_name agent.yourdomain.com;

    location / {
        proxy_pass http://127.0.0.1:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # Critical for SSE streaming
        proxy_buffering off;
        proxy_cache off;
        proxy_read_timeout 300s;
        proxy_send_timeout 300s;
    }
}
EOF

sudo ln -s /etc/nginx/sites-available/research-agent /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx

# Get HTTPS certificate
sudo certbot --nginx -d agent.yourdomain.com --non-interactive --agree-tos

Best Practices for Self-Hosting

Model 3: Serverless Functions

What It Is

Serverless platforms like AWS Lambda, Google Cloud Functions, Azure Container Apps, or Cloudflare Workers execute your code in ephemeral containers that spin up on demand and disappear after the response. You pay only for the milliseconds of actual execution. The platform handles scaling, load balancing, and availability automatically.

Why It Matters

The Serverless Challenge: Timeouts and State

Standard serverless functions have hard timeout limits: AWS Lambda caps at 15 minutes (900 seconds), but many providers like Cloudflare Workers cap at 30 seconds. For AI agents that need multiple LLM round-trips, these limits are dangerous. Additionally, serverless functions are stateless by design—you cannot rely on in-memory state between invocations. This fundamentally changes how you architect the agent.

Complete Deployment Example (AWS Lambda + Function URL)

We'll deploy the agent to AWS Lambda using a container image (not the ZIP upload method, because our dependencies exceed the 250 MB limit). We'll expose it via a Lambda Function URL with streaming support. 

# lambda_handler.py - Entry point for AWS Lambda
import os
import json
import asyncio
from openai import OpenAI

# Lambda runs in a single-threaded environment by default.
# We'll use the synchronous version of our agent.
from agent_core import run_agent_sync

def lambda_handler(event, context):
    """
    AWS Lambda handler for the research agent.
    Event format follows API Gateway / Function URL conventions.
    """
    try:
        # Parse the request body
        body = json.loads(event.get("body", "{}"))
        query = body.get("query", "")
        
        if not query:
            return {
                "statusCode": 400,
                "headers": {"Content-Type": "application/json"},
                "body": json.dumps({"error": "Missing 'query' field"})
            }
        
        # Run the agent synchronously
        result = run_agent_sync(query)
        
        return {
            "statusCode": 200,
            "headers": {
                "Content-Type": "application/json",
                "Cache-Control": "no-cache"
            },
            "body": json.dumps({"answer": result})
        }
    
    except Exception as e:
        print(f"Agent execution failed: {str(e)}")
        return {
            "statusCode": 500,
            "headers": {"Content-Type": "application/json"},
            "body": json.dumps({"error": f"Agent error: {str(e)}"})
        }

# For streaming responses, we need a different approach.
# Lambda Function URLs support response streaming via the InvokeWithResponseStream API.
# Here's a streaming-compatible handler using the runtime's built-in stream support:

def lambda_handler_streaming(event, context):
    """
    Streaming handler that flushes chunks as they arrive.
    Requires: AWS Lambda runtime with streaming support (Python 3.12+)
    """
    import io
    body = json.loads(event.get("body", "{}"))
    query = body.get("query", "What is an AI agent?")
    
    client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))
    messages = [
        {"role": "system", "content": "You are a helpful research assistant."},
        {"role": "user", "content": query}
    ]
    
    # Use streaming response from OpenAI
    response_stream = client.chat.completions.create(
        model="gpt-4o",
        messages=messages,
        stream=True
    )
    
    # Build a streaming response
    def stream_generator():
        yield json.dumps({"event": "started"}) + "\n"
        for chunk in response_stream:
            if chunk.choices[0].delta.content:
                yield json.dumps({
                    "event": "thought", 
                    "text": chunk.choices[0].delta.content
                }) + "\n"
        yield json.dumps({"event": "done"}) + "\n"
    
    # Return a streaming response
    return {
        "statusCode": 200,
        "headers": {
            "Content-Type": "application/x-ndjson",
            "Transfer-Encoding": "chunked"
        },
        "body": stream_generator()
    }

Dockerfile optimized for Lambda:

# Dockerfile.lambda
FROM public.ecr.aws/lambda/python:3.12

WORKDIR ${LAMBDA_TASK_ROOT}

# Copy application code
COPY agent_core.py ${LAMBDA_TASK_ROOT}/agent_core.py
COPY lambda_handler.py ${LAMBDA_TASK_ROOT}/lambda_handler.py

# Install dependencies
COPY requirements-lambda.txt .
RUN pip install --no-cache-dir -r requirements-lambda.txt

# Lambda uses the CMD to locate the handler
CMD ["lambda_handler.lambda_handler"]

# requirements-lambda.txt
openai==1.55.0
httpx==0.28.1

Deployment script using AWS CLI:

#!/bin/bash
# deploy_lambda.sh - Deploy the agent to AWS Lambda

AWS_REGION="us-east-1"
FUNCTION_NAME="research-agent-lambda"
ROLE_ARN="arn:aws:iam::123456789012:role/lambda-execution-role"

# 1. Build the Docker image for Lambda
docker build -t research-agent-lambda:v1 -f Dockerfile.lambda .

# 2. Create an ECR repository (skip if exists)
aws ecr create-repository \
    --repository-name research-agent-lambda \
    --region $AWS_REGION 2>/dev/null || true

# 3. Authenticate Docker to ECR
aws ecr get-login-password --region $AWS_REGION | \
    docker login --username AWS --password-stdin \
    123456789012.dkr.ecr.$AWS_REGION.amazonaws.com

# 4. Tag and push the image
docker tag research-agent-lambda:v1 \
    123456789012.dkr.ecr.$AWS_REGION.amazonaws.com/research-agent-lambda:v1

docker push 123456789012.dkr.ecr.$AWS_REGION.amazonaws.com/research-agent-lambda:v1

# 5. Create or update the Lambda function
aws lambda create-function \
    --function-name $FUNCTION_NAME \
    --package-type Image \
    --code ImageUri=123456789012.dkr.ecr.$AWS_REGION.amazonaws.com/research-agent-lambda:v1 \
    --role $ROLE_ARN \
    --timeout 900 \
    --memory-size 2048 \
    --environment-variables "OPENAI_API_KEY=sk-your-key-here" \
    --region $AWS_REGION 2>/dev/null || \
    
aws lambda update-function-code \
    --function-name $FUNCTION_NAME \
    --image-uri 123456789012.dkr.ecr.$AWS_REGION.amazonaws.com/research-agent-lambda:v1 \
    --region $AWS_REGION

# 6. Create a Function URL (public HTTPS endpoint)
aws lambda create-url-config \
    --function-name $FUNCTION_NAME \
    --auth-type NONE \
    --invoke-mode RESPONSE_STREAM \
    --region $AWS_REGION

# 7. Get the Function URL
aws lambda get-url-config \
    --function-name $FUNCTION_NAME \
    --region $AWS_REGION \
    --query 'FunctionUrl' --output text

The critical detail here is --timeout 900—we're setting the Lambda timeout to the maximum 15 minutes. For agents that may run longer, Lambda alone won't suffice; you'd need to decompose the agent into a step function workflow. Also note --invoke-mode RESPONSE_STREAM which enables streaming responses through the Function URL, essential for real-time agent feedback.

Best Practices for Serverless AI Agents

Side-by-Side Comparison

Here is a summary table to help you choose based on your specific constraints:

Criterion Managed Self-Hosted Serverless
Setup Time Minutes (single CLI command) Hours (Docker, Nginx, TLS, monitoring) 30–60 minutes (ECR push, IAM roles)
Max Execution Time Configurable, typically 15 min+ Unlimited (set by you) Provider-limited (15 min on Lambda, 30s on Workers)
Cold Start Latency 2–5s (mitigated by warm pools) 0s (always running) 3–8s (mitigated by provisioned concurrency)
Cost Model Per compute second + idle warm fee Fixed monthly (VPS/instance cost) Per millisecond of execution only
Scaling Limit Platform-defined (often 100+ concurrent) Hardware-limited (you scale manually) Cloud-defined (1000+ concurrent for Lambda)
State Management Platform-provided (Redis, DB, or disk mounts) Full control (volumes, external DBs) Must be fully externalized
Streaming Support First-class (SSE, WebSocket) Full control (Nginx tuning required) Varies (Lambda supports streaming; Workers limited)
Data Sovereignty Data flows through platform infra Complete control over data path Data stays within your cloud VPC

When to Choose Each Model

Choose Managed when you're an individual developer or a small team that wants to ship an AI agent quickly without building infrastructure. Managed platforms excel at developer experience and time-to-market. They're also ideal when you need GPU access for local models but don't want to manage CUDA drivers and container registries yourself.

Choose Self-Hosted when you have strict compliance requirements, need guaranteed uptime with no cold starts, or expect consistent high traffic that makes fixed-cost VPS more economical than per-request billing. Self-hosting also makes sense when your agent requires specialized hardware, custom network configurations, or integration with on-premises systems.

Choose Serverless when your agent workload is sporadic (bursts of requests with long idle periods), you need massive horizontal scaling without manual intervention, or you're already deeply invested in a cloud ecosystem and want tight integration with services like SQS, DynamoDB, and Step Functions. Serverless works best for agents that complete within the timeout limit and don't require persistent in-memory state.

Hybrid Architectures: The Pragmatic Reality

In production, many teams end up with a hybrid approach. A common pattern is:

— Ad —

Google AdSense will appear here after approval

← Back to all articles