Deploying Local Models as AWS Lambda Functions

Deploying Local Models as AWS Lambda Functions

Machine learning models are powerful, but deploying them in production can be complex. While many developers turn to dedicated ML platforms like SageMaker or containerized services like ECS, AWS Lambda offers a compelling alternative for lightweight, event-driven inference workloads. By packaging your locally trained or fine-tuned models into Lambda functions, you can serve predictions without managing servers, scaling infrastructure, or paying for idle compute time.

What Is It?

Deploying local models as AWS Lambda functions means taking a machine learning model you've trained or downloaded locally — such as a scikit-learn classifier, a PyTorch model, an ONNX runtime model, or a lightweight transformer — and packaging it together with its inference code into a deployable artifact that AWS Lambda can execute. When invoked, the Lambda function loads the model into memory, processes the input, and returns a prediction.

This approach treats ML inference as a standard serverless function. The model artifacts (weights, tokenizer, configuration files) are bundled into the deployment package or stored in Amazon S3 and loaded at runtime. The function exposes an HTTP endpoint via API Gateway or can be triggered by other AWS services like S3, SQS, or EventBridge.

Why It Matters

Serverless ML inference has several distinct advantages that make it attractive for many real-world scenarios:

• Cost efficiency: You pay only for actual invocations and execution time. There's no cost for idle time, which is ideal for models with sporadic or low-volume traffic.
• Automatic scaling: Lambda handles concurrency automatically. Whether you receive one request per hour or a thousand per second, AWS manages the scaling without intervention.
• Zero infrastructure management: No servers to patch, no containers to orchestrate, no load balancers to configure. You focus entirely on your model and inference logic.
• Event-driven architectures: Lambda integrates natively with the AWS ecosystem, making it easy to trigger inference from file uploads, database changes, queue messages, or scheduled events.
• Rapid prototyping: You can deploy a model in minutes and iterate quickly, which is valuable for experimentation and A/B testing.

However, Lambda does have constraints. The maximum deployment package size is 250 MB (unzipped, including layers), the maximum memory is 10 GB, and the maximum execution timeout is 15 minutes. These limits make Lambda best suited for smaller models and inference tasks that complete within seconds rather than long-running training or heavy batch processing.

Prerequisites

Before you begin, ensure you have the following:

• An AWS account with appropriate permissions to create Lambda functions, IAM roles, and optionally S3 buckets
• The AWS CLI installed and configured with your credentials
• Python 3.11 or later installed locally
• A trained or downloaded model ready for deployment
• Basic familiarity with Python and AWS Lambda concepts

Step 1: Preparing Your Model

The first step is selecting and preparing a model that fits within Lambda's constraints. For this tutorial, we'll use a scikit-learn model as a straightforward example, but the same principles apply to PyTorch, ONNX, or TensorFlow Lite models.

Let's start by training a simple text classification model locally and saving it:

import joblib
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline

# Sample training data
texts = [
    "I love this product, it works great",
    "Terrible experience, would not recommend",
    "Amazing quality and fast delivery",
    "Worst purchase I have ever made",
    "Highly satisfied with the service",
    "Complete waste of money",
    "Best decision I ever made",
    "Disappointed with the quality"
]
labels = [1, 0, 1, 0, 1, 0, 1, 0]  # 1 = positive, 0 = negative

# Create and train the pipeline
pipeline = Pipeline([
    ('vectorizer', TfidfVectorizer(max_features=1000)),
    ('classifier', LogisticRegression())
])

pipeline.fit(texts, labels)

# Save the model
joblib.dump(pipeline, 'model.joblib')
print("Model saved successfully")

After running this script, you'll have a model.joblib file. This is your model artifact that will be packaged with the Lambda function.

Step 2: Creating the Lambda Handler

The Lambda handler is the entry point for your function. It receives the event payload, loads the model, performs inference, and returns the result. A well-structured handler separates model loading from inference logic to take advantage of Lambda's execution environment reuse.

import json
import joblib
import os

# Global variable to hold the loaded model
# This persists across warm invocations
model = None

def load_model():
    """Load the model from the deployment package."""
    global model
    if model is None:
        model_path = os.path.join(os.path.dirname(__file__), 'model.joblib')
        model = joblib.load(model_path)
    return model

def handler(event, context):
    """
    Lambda handler function.
    Expects an event with a 'text' field in the body.
    Returns a sentiment prediction.
    """
    try:
        # Parse the incoming event
        if isinstance(event.get('body'), str):
            body = json.loads(event['body'])
        else:
            body = event.get('body', event)
        
        text = body.get('text', '')
        
        if not text:
            return {
                'statusCode': 400,
                'headers': {'Content-Type': 'application/json'},
                'body': json.dumps({'error': 'No text provided'})
            }
        
        # Load model (uses cached version on warm starts)
        model = load_model()
        
        # Perform inference
        prediction = model.predict([text])[0]
        probabilities = model.predict_proba([text])[0]
        
        result = {
            'text': text,
            'sentiment': 'positive' if prediction == 1 else 'negative',
            'confidence': float(max(probabilities))
        }
        
        return {
            'statusCode': 200,
            'headers': {'Content-Type': 'application/json'},
            'body': json.dumps(result)
        }
    
    except Exception as e:
        return {
            'statusCode': 500,
            'headers': {'Content-Type': 'application/json'},
            'body': json.dumps({'error': str(e)})
        }

Notice that the model is loaded into a global variable outside the handler function. This is a critical optimization: on the first invocation (cold start), the model is loaded from disk. On subsequent invocations (warm starts), the model is already in memory, dramatically reducing latency.

Step 3: Packaging the Deployment

Lambda deployment packages for Python can be created as ZIP archives containing your code and dependencies. For models with native dependencies like scikit-learn, you need to install the packages targeting the Lambda runtime environment (Amazon Linux 2).

Create a directory structure for your deployment package:

mkdir lambda-deployment
cd lambda-deployment

# Create the function directory
mkdir -p function

# Copy your handler and model
cp ../handler.py function/
cp ../model.joblib function/

# Install dependencies targeting Lambda's environment
pip install scikit-learn joblib numpy scipy \
    --platform manylinux2014_x86_64 \
    --only-binary=:all: \
    --target function/

# Create the deployment ZIP
cd function
zip -r ../deployment.zip .
cd ..

The resulting deployment.zip file contains your handler code, the model artifact, and all required Python dependencies. Verify that the total unzipped size stays under 250 MB to comply with Lambda's limits.

Step 4: Deploying with the AWS CLI

With your deployment package ready, you can create the Lambda function using the AWS CLI. First, create an IAM role that grants Lambda basic execution permissions:

# Create a trust policy file
cat > trust-policy.json << 'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF

# Create the IAM role
aws iam create-role \
    --role-name lambda-model-role \
    --assume-role-policy-document file://trust-policy.json

# Attach the basic execution policy
aws iam attach-role-policy \
    --role-name lambda-model-role \
    --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

Now create the Lambda function:

# Get your AWS account ID
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

# Create the Lambda function
aws lambda create-function \
    --function-name sentiment-model \
    --runtime python3.11 \
    --role arn:aws:iam::${ACCOUNT_ID}:role/lambda-model-role \
    --handler handler.handler \
    --zip-file fileb://deployment.zip \
    --timeout 30 \
    --memory-size 1024

Key parameters to note: --timeout sets the maximum execution time (30 seconds is reasonable for inference), and --memory-size allocates both memory and CPU (Lambda allocates CPU proportionally to memory, so 1024 MB gives you a full vCPU).

Step 5: Testing the Function

You can test the function directly using the AWS CLI with a test event:

# Create a test event
cat > test-event.json << 'EOF'
{
  "body": "{\"text\": \"This product exceeded all my expectations!\"}"
}
EOF

# Invoke the function
aws lambda invoke \
    --function-name sentiment-model \
    --payload file://test-event.json \
    --cli-binary-format raw-in-base64-out \
    response.json

# View the response
cat response.json

The response should look something like this:

{
  "statusCode": 200,
  "headers": {"Content-Type": "application/json"},
  "body": "{\"text\": \"This product exceeded all my expectations!\", \"sentiment\": \"positive\", \"confidence\": 0.87}"
}

Step 6: Exposing via API Gateway

To make your model accessible over HTTP, attach an API Gateway endpoint. The simplest way is to use a Lambda Function URL, which provides a dedicated HTTPS endpoint without needing to configure a full API Gateway:

aws lambda create-function-url-config \
    --function-name sentiment-model \
    --auth-type AWS_IAM

For public access without IAM authentication, use --auth-type NONE and add a resource-based policy. Alternatively, for production workloads, use API Gateway for more control over routing, throttling, and authentication:

# Create a REST API
API_ID=$(aws apigateway create-rest-api \
    --name 'sentiment-api' \
    --endpoint-configuration '{"types": ["REGIONAL"]}' \
    --query 'id' --output text)

# Get the root resource ID
ROOT_ID=$(aws apigateway get-resources \
    --rest-api-id ${API_ID} \
    --query 'items[0].id' --output text)

# Create a POST method
aws apigateway put-method \
    --rest-api-id ${API_ID} \
    --resource-id ${ROOT_ID} \
    --http-method POST \
    --authorization-type NONE

# Integrate with Lambda
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
aws apigateway put-integration \
    --rest-api-id ${API_ID} \
    --resource-id ${ROOT_ID} \
    --http-method POST \
    --type AWS_PROXY \
    --integration-http-method POST \
    --uri arn:aws:apigateway:us-east-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-east-1:${ACCOUNT_ID}:function:sentiment-model/invocations

# Deploy the API
aws apigateway create-deployment \
    --rest-api-id ${API_ID} \
    --stage-name prod

echo "API endpoint: https://${API_ID}.execute-api.us-east-1.amazonaws.com/prod"

You can now send POST requests to your endpoint:

curl -X POST \
    https://your-api-id.execute-api.us-east-1.amazonaws.com/prod \
    -H "Content-Type: application/json" \
    -d '{"text": "I am very happy with this purchase"}'

Deploying Larger Models with Lambda Layers and S3

When your model or dependencies exceed the 250 MB limit, you can use Lambda Layers for dependencies and load model artifacts from S3 at runtime. This pattern is essential for deploying larger models like lightweight transformers.

First, upload your model to S3:

aws s3 mb s3://my-model-bucket
aws s3 cp model.joblib s3://my-model-bucket/models/model.joblib

Then modify your handler to download the model on cold starts:

import json
import joblib
import os
import boto3
import tempfile

model = None

def load_model():
    global model
    if model is None:
        s3 = boto3.client('s3')
        with tempfile.NamedTemporaryFile(suffix='.joblib', delete=False) as tmp:
            s3.download_file(
                'my-model-bucket',
                'models/model.joblib',
                tmp.name
            )
            model = joblib.load(tmp.name)
        os.unlink(tmp.name)
    return model

def handler(event, context):
    try:
        body = json.loads(event.get('body', '{}')) if isinstance(event.get('body'), str) else event.get('body', event)
        text = body.get('text', '')
        
        if not text:
            return {
                'statusCode': 400,
                'body': json.dumps({'error': 'No text provided'})
            }
        
        model = load_model()
        prediction = model.predict([text])[0]
        
        return {
            'statusCode': 200,
            'body': json.dumps({
                'sentiment': 'positive' if prediction == 1 else 'negative'
            })
        }
    except Exception as e:
        return {
            'statusCode': 500,
            'body': json.dumps({'error': str(e)})
        }

Deploying PyTorch and ONNX Models

For deep learning models, ONNX Runtime is often the best choice for Lambda because it's lightweight and optimized for inference. Here's an example handler for an ONNX model:

import json
import numpy as np
import onnxruntime as ort
import os

session = None

def get_session():
    global session
    if session is None:
        model_path = os.path.join(os.path.dirname(__file__), 'model.onnx')
        session = ort.InferenceSession(model_path)
    return session

def handler(event, context):
    try:
        body = json.loads(event.get('body', '{}')) if isinstance(event.get('body'), str) else event.get('body', event)
        
        # Convert input to numpy array
        input_data = np.array(body.get('features', []), dtype=np.float32)
        
        if input_data.size == 0:
            return {
                'statusCode': 400,
                'body': json.dumps({'error': 'No features provided'})
            }
        
        # Ensure correct shape
        if input_data.ndim == 1:
            input_data = input_data.reshape(1, -1)
        
        session = get_session()
        input_name = session.get_inputs()[0].name
        outputs = session.run(None, {input_name: input_data})
        
        # Process outputs
        predictions = outputs[0].tolist()
        
        return {
            'statusCode': 200,
            'body': json.dumps({'predictions': predictions})
        }
    except Exception as e:
        return {
            'statusCode': 500,
            'body': json.dumps({'error': str(e)})
        }

Best Practices

Optimize Cold Start Performance

Cold starts are the latency penalty incurred when Lambda provisions a new execution environment. For ML models, cold starts are dominated by model loading time. To minimize them:

• Keep models as small as possible by quantizing weights, pruning, or using efficient formats like ONNX
• Load the model at module level (outside the handler) so it happens once per container, not per invocation
• Use provisioned concurrency for latency-sensitive workloads to keep environments warm
• Avoid importing unnecessary libraries at the top level; use lazy imports for rarely used modules

Right-Size Memory Allocation

Memory allocation in Lambda also determines CPU power. A model that runs slowly at 128 MB might run significantly faster at 1024 MB or 2048 MB, and the reduced execution time can actually lower total cost. Benchmark your function at different memory settings:

# Test with different memory configurations
for memory in 512 1024 2048 4096; do
    aws lambda update-function-configuration \
        --function-name sentiment-model \
        --memory-size ${memory}
    
    sleep 10
    
    echo "Testing with ${memory} MB..."
    aws lambda invoke \
        --function-name sentiment-model \
        --payload file://test-event.json \
        --cli-binary-format raw-in-base64-out \
        /dev/null
done

Use Lambda Layers for Shared Dependencies

If you deploy multiple Lambda functions with the same ML dependencies, package those dependencies as a Lambda Layer. Layers can be shared across functions and reduce deployment package sizes. They also let you update dependencies independently from your application code.

# Create a layer with ML dependencies
mkdir -p layer/python
pip install scikit-learn joblib numpy scipy \
    --platform manylinux2014_x86_64 \
    --only-binary=:all: \
    --target layer/python/

cd layer
zip -r ../ml-deps-layer.zip python/
cd ..

aws lambda publish-layer-version \
    --layer-name ml-dependencies \
    --zip-file fileb://ml-deps-layer.zip \
    --compatible-runtimes python3.11

# Attach the layer to your function
LAYER_ARN=$(aws lambda list-layers \
    --query 'Layers[?LayerName==`ml-dependencies`].LatestMatchingLayerVersion.Arn' \
    --output text)

aws lambda update-function-configuration \
    --function-name sentiment-model \
    --layers ${LAYER_ARN}

Implement Proper Error Handling and Logging

Production ML endpoints need robust error handling. Validate inputs, catch model-specific exceptions, and log meaningful information for debugging:

import json
import logging
import traceback

logger = logging.getLogger()
logger.setLevel(logging.INFO)

def handler(event, context):
    try:
        # Log incoming request (sanitize sensitive data)
        logger.info(f"Received request: {json.dumps(event)[:500]}")
        
        body = json.loads(event.get('body', '{}')) if isinstance(event.get('body'), str) else event.get('body', event)
        
        # Input validation
        text = body.get('text', '')
        if not isinstance(text, str):
            raise ValueError("Input 'text' must be a string")
        if len(text) > 10000:
            raise ValueError("Input text exceeds maximum length of 10000 characters")
        if not text.strip():
            return {
                'statusCode': 400,
                'body': json.dumps({'error': 'Text cannot be empty'})
            }
        
        # Inference
        model = load_model()
        prediction = model.predict([text])[0]
        confidence = float(max(model.predict_proba([text])[0]))
        
        logger.info(f"Prediction: {prediction}, Confidence: {confidence:.4f}")
        
        return {
            'statusCode': 200,
            'headers': {
                'Content-Type': 'application/json',
                'Access-Control-Allow-Origin': '*'
            },
            'body': json.dumps({
                'sentiment': 'positive' if prediction == 1 else 'negative',
                'confidence': confidence
            })
        }
    
    except ValueError as e:
        logger.warning(f"Validation error: {str(e)}")
        return {
            'statusCode': 400,
            'body': json.dumps({'error': str(e)})
        }
    except Exception as e:
        logger.error(f"Unexpected error: {traceback.format_exc()}")
        return {
            'statusCode': 500,
            'body': json.dumps({'error': 'Internal server error'})
        }

Monitor with CloudWatch and X-Ray

Set up CloudWatch alarms for error rates, latency, and throttling. Use AWS X-Ray to trace requests through your inference pipeline and identify bottlenecks. Track custom metrics like model confidence distributions to detect data drift:

import boto3
import time

cloudwatch = boto3.client('cloudwatch')

def put_metric(name, value, unit='None'):
    cloudwatch.put_metric_data(
        Namespace='MLModel/sentiment',
        MetricData=[{
            'MetricName': name,
            'Value': value,
            'Unit': unit,
            'Timestamp': time.time()
        }]
    )

# In your handler, after inference:
put_metric('InferenceLatency', duration_ms, 'Milliseconds')
put_metric('AverageConfidence', confidence, 'None')
put_metric('PositivePredictions', 1 if prediction == 1 else 0, 'Count')

Consider Container Image Deployment for Complex Models

When your deployment exceeds 250 MB but stays under 10 GB, use Lambda's container image support. This allows you to package larger models and use custom Dockerfiles with system-level dependencies:

# Dockerfile
FROM public.ecr.aws/lambda/python:3.11

# Install system dependencies
RUN yum install -y libgomp

# Copy requirements and install
COPY requirements.txt .
RUN pip install -r requirements.txt

# Copy model and handler
COPY model.joblib ${LAMBDA_TASK_ROOT}/
COPY handler.py ${LAMBDA_TASK_ROOT}/

# Set the handler
CMD ["handler.handler"]

Build and push the image:

# Build the image
docker build -t sentiment-model .

# Authenticate with ECR
aws ecr get-login-password | docker login --username AWS --password-stdin ${ACCOUNT_ID}.dkr.ecr.us-east-1.amazonaws.com

# Create ECR repository
aws ecr create-repository --repository-name sentiment-model

# Tag and push
docker tag sentiment-model:latest ${ACCOUNT_ID}.dkr.ecr.us-east-1.amazonaws.com/sentiment-model:latest
docker push ${ACCOUNT_ID}.dkr.ecr.us-east-1.amazonaws.com/sentiment-model:latest

# Create or update the Lambda function with the container image
aws lambda create-function \
    --function-name sentiment-model \
    --package-type Image \
    --code ImageUri=${ACCOUNT_ID}.dkr.ecr.us-east-1.amazonaws.com/sentiment-model:latest \
    --role arn:aws:iam::${ACCOUNT_ID}:role/lambda-model-role \
    --timeout 60 \
    --memory-size 2048

Version Control and Blue-Green Deployments

Use Lambda versions and aliases to manage deployments safely. Publish a new version when you update your model, and use aliases to route traffic. This enables blue-green deployments and instant rollback if a new model version performs poorly:

# Publish a new version
aws lambda publish-version \
    --function-name sentiment-model \
    --description "Model v2 with improved accuracy"

# Update the alias to point to the new version
aws lambda update-alias \
    --function-name sentiment-model \
    --name prod \
    --function-version 2

# For canary deployments, use weighted routing
aws lambda update-alias \
    --function-name sentiment-model \
    --name prod \
    --routing-config '{"AdditionalVersionWeights": {"2": 0.1}}'

Conclusion

Deploying local models as AWS Lambda functions provides a serverless, cost-effective, and scalable approach to serving ML predictions. By carefully packaging your model artifacts, optimizing cold start performance, and following best practices around memory sizing, error handling, and monitoring, you can build production-grade inference endpoints that scale automatically with demand. While Lambda's size and timeout constraints make it unsuitable for every ML workload, it excels for lightweight models, event-driven inference, and scenarios where cost efficiency and operational simplicity are paramount. As your models grow in complexity, the same patterns extend naturally to container-based Lambda deployments, giving you a flexible path from prototype to production without changing your core architecture.