Scaling SNS: From Prototype to Production

Introduction to Scaling SNS

Amazon Simple Notification Service (SNS) is a fully managed pub/sub messaging service that enables you to decouple microservices, distribute notifications, and fan-out messages to multiple subscribers. While building a prototype with SNS is straightforward—create a topic, subscribe an endpoint, publish a message—scaling that prototype to production introduces a host of challenges around throughput, reliability, cost, and observability.

This tutorial walks through the journey of taking an SNS-based architecture from a quick prototype to a robust, production-grade system. We will cover the fundamentals, identify common scaling bottlenecks, and provide practical code examples and best practices along the way.

What Is Amazon SNS?

Amazon SNS is a push-based messaging platform that supports two primary patterns: application-to-application (A2A) pub/sub and application-to-person (A2P) notifications. In the A2A model, a publisher sends messages to a topic, and SNS fans those messages out to subscribed endpoints such as AWS Lambda, SQS queues, HTTP/S webhooks, email, or mobile push services.

Key characteristics of SNS include:

At-least-once delivery: SNS attempts to deliver each message at least once, meaning subscribers must be idempotent.
High throughput by default: Standard topics support up to 300,000 publish requests per second per account across all standard topics in most regions.
Multiple protocols: SQS, Lambda, HTTP/S, email, SMS, and mobile push are all supported as subscription endpoints.
FIFO topics: For ordering and deduplication, FIFO topics provide exactly-once processing when paired with FIFO SQS queues.

Why Scaling SNS Matters

In a prototype, you might publish a few hundred messages per minute to a single topic with one or two subscribers. Everything works fine. But as your application grows, several issues emerge:

Subscriber bottlenecks: A slow HTTP endpoint can cause SNS retries to pile up, leading to increased latency and potential message loss if retries are exhausted.
Cost explosion: Publishing large payloads instead of references, or using SMS/email subscriptions indiscriminately, can inflate costs dramatically.
Operational blind spots: Without metrics and dead-letter queues, failed deliveries go unnoticed.
Security gaps: Prototype topics are often overly permissive, which becomes a liability at scale.
Throttling on downstream services: Fanning out to Lambda or HTTP endpoints without concurrency controls can overwhelm consumers.

Scaling SNS is less about SNS itself—AWS handles the infrastructure—and more about how you design the surrounding system: subscription patterns, message payloads, error handling, and observability.

From Prototype to Production: The Architecture

The Prototype

A typical prototype looks like this: a single SNS topic receives events from an application, and a Lambda function is subscribed directly to process them. Here is a minimal example using the AWS SDK for Python (Boto3):

import boto3

sns = boto3.client('sns', region_name='us-east-1')

# Create a topic
topic = sns.create_topic(Name='orders-topic')
topic_arn = topic['TopicArn']

# Subscribe a Lambda function
sns.subscribe(
    TopicArn=topic_arn,
    Protocol='lambda',
    Endpoint='arn:aws:lambda:us-east-1:123456789012:function:process-order'
)

# Publish a message
sns.publish(
    TopicArn=topic_arn,
    Message='{"orderId": "12345", "status": "created"}'
)

This works for a demo, but it has problems: no error handling, no dead-letter queue, no batching, and no visibility into failures. Let's evolve it.

Production Architecture

In production, the recommended pattern is to use SNS as a fan-out router with SQS queues as buffers between SNS and your consumers. This decouples message production from consumption, absorbs traffic spikes, and enables retry logic per consumer.

import boto3

sns = boto3.client('sns')
sqs = boto3.client('sqs')

# 1. Create the SNS topic
topic = sns.create_topic(Name='orders-prod')
topic_arn = topic['TopicArn']

# 2. Create an SQS queue with a dead-letter queue
dlq = sqs.create_queue(QueueName='orders-dlq')
dlq_arn = sqs.get_queue_attributes(
    QueueUrl=dlq['QueueUrl'],
    AttributeNames=['QueueArn']
)['Attributes']['QueueArn']

main_queue = sqs.create_queue(
    QueueName='orders-queue',
    Attributes={
        'RedrivePolicy': f'{{"deadLetterTargetArn":"{dlq_arn}","maxReceiveCount":"5"}}',
        'VisibilityTimeout': '60'
    }
)
main_queue_arn = sqs.get_queue_attributes(
    QueueUrl=main_queue['QueueUrl'],
    AttributeNames=['QueueArn']
)['Attributes']['QueueArn']

# 3. Allow SNS to send messages to the SQS queue
sqs.set_queue_attributes(
    QueueUrl=main_queue['QueueUrl'],
    Attributes={
        'Policy': json.dumps({
            "Version": "2012-10-17",
            "Statement": [{
                "Effect": "Allow",
                "Principal": {"Service": "sns.amazonaws.com"},
                "Action": "sqs:SendMessage",
                "Resource": main_queue_arn,
                "Condition": {
                    "ArnEquals": {"aws:SourceArn": topic_arn}
                }
            }]
        })
    }
)

# 4. Subscribe the SQS queue to the SNS topic
sns.subscribe(
    TopicArn=topic_arn,
    Protocol='sqs',
    Endpoint=main_queue_arn
)

With this setup, SNS fans out messages to SQS, which acts as a durable buffer. Your consumer (Lambda, ECS, or EC2 worker) pulls from SQS at its own pace. Failed messages after five retries move to the DLQ for manual inspection.

Optimizing Message Publishing

Batch Publishing

For high-throughput workloads, use the publish_batch API to send up to 10 messages in a single request. This reduces API calls, lowers cost, and improves throughput.

import boto3

sns = boto3.client('sns')
topic_arn = 'arn:aws:sns:us-east-1:123456789012:orders-prod'

messages = [
    {"id": f"msg-{i}", "orderId": str(i), "status": "created"}
    for i in range(10)
]

response = sns.publish_batch(
    TopicArn=topic_arn,
    PublishBatchRequestEntries=[
        {
            'Id': msg['id'],
            'Message': json.dumps(msg),
            'MessageAttributes': {
                'eventType': {
                    'DataType': 'String',
                    'StringValue': 'order.created'
                }
            }
        }
        for msg in messages
    ]
)

print(f"Success: {len(response.get('Successful', []))}")
print(f"Failed: {len(response.get('Failed', []))}")

Message Filtering

When multiple subscribers are attached to one topic, use message attribute filtering so each subscriber only receives relevant messages. This reduces unnecessary processing and cost.

# Subscribe with a filter policy
sns.subscribe(
    TopicArn=topic_arn,
    Protocol='sqs',
    Endpoint=queue_arn,
    Attributes={
        'FilterPolicy': json.dumps({
            "eventType": ["order.created", "order.updated"]
        })
    },
    ReturnSubscriptionArn=True
)

With this filter, the subscriber only receives messages where the eventType attribute matches order.created or order.updated. Messages with other event types are never delivered to this queue, saving downstream compute.

Handling Payload Size Limits

SNS has a 256 KB payload limit per message. In production, you should avoid sending large payloads through SNS entirely. Instead, use the claim-check pattern: store the full payload in S3 and send a reference through SNS.

import boto3
import json
import uuid

s3 = boto3.client('s3')
sns = boto3.client('sns')

def publish_large_event(topic_arn, event_data, bucket_name):
    # Store the full payload in S3
    object_key = f"events/{uuid.uuid4()}.json"
    s3.put_object(
        Bucket=bucket_name,
        Key=object_key,
        Body=json.dumps(event_data).encode('utf-8')
    )

    # Publish a lightweight reference message
    sns.publish(
        TopicArn=topic_arn,
        Message=json.dumps({
            'bucket': bucket_name,
            'key': object_key,
            'eventType': event_data.get('eventType')
        }),
        MessageAttributes={
            'eventType': {
                'DataType': 'String',
                'StringValue': event_data.get('eventType', 'unknown')
            }
        }
    )

The consumer retrieves the full payload from S3 when processing. This keeps SNS fast and cheap while supporting arbitrarily large payloads.

Observability and Monitoring

At scale, you need visibility into publish rates, delivery failures, and subscription health. Enable SNS topic metrics and set up CloudWatch alarms for critical signals.

import boto3

cloudwatch = boto3.client('cloudwatch')

# Alarm for delivery failures
cloudwatch.put_metric_alarm(
    AlarmName='sns-orders-delivery-failures',
    AlarmDescription='Alert when SNS delivery failures exceed threshold',
    MetricName='NumberOfNotificationsFailed',
    Namespace='AWS/SNS',
    Statistic='Sum',
    Period=300,
    EvaluationPeriods=1,
    Threshold=10,
    ComparisonOperator='GreaterThanThreshold',
    Dimensions=[
        {'Name': 'TopicName', 'Value': 'orders-prod'}
    ],
    AlarmActions=[
        'arn:aws:sns:us-east-1:123456789012:ops-alerts'
    ]
)

Key metrics to monitor include NumberOfMessagesPublished, NumberOfNotificationsDelivered, NumberOfNotificationsFailed, and PublishSize. Also enable delivery status logging for HTTP/S subscriptions to capture detailed failure reasons.

Best Practices for Production SNS

Always use SQS as a buffer: Never subscribe HTTP endpoints or Lambda directly to SNS for critical workloads. SQS provides retries, visibility timeouts, and DLQs.
Design consumers for idempotency: SNS provides at-least-once delivery. Use message deduplication IDs (with FIFO topics) or application-level idempotency keys.
Keep payloads small: Send references, not full payloads. Use the claim-check pattern for anything over a few kilobytes.
Use message filtering aggressively: Filter at the subscription level to avoid unnecessary processing and reduce costs.
Enable dead-letter queues: Configure DLQs on both the SQS queue and the SNS subscription to capture undeliverable messages.
Secure your topics: Use least-privilege IAM policies, enable server-side encryption with KMS, and restrict cross-account access with topic policies.
Use FIFO topics when ordering matters: FIFO topics with content-based deduplication and message groups provide ordering guarantees, though at lower throughput (300 TPS per topic by default).
Batch your publishes: Use publish_batch to reduce API calls and cost by up to 10x for high-volume workloads.
Tag your resources: Apply cost allocation tags to topics for accurate billing attribution across teams and projects.

Securing SNS at Scale

Security becomes critical as your topic count and subscriber base grow. Start by enabling KMS encryption on topics containing sensitive data:

sns = boto3.client('sns')

sns.create_topic(
    Name='sensitive-events',
    Attributes={
        'KmsMasterKeyId': 'arn:aws:kms:us-east-1:123456789012:key/your-key-id'
    }
)

Apply a topic access policy that restricts publishing to specific IAM roles and limits cross-account subscriptions:

policy = {
    "Version": "2012-10-17",
    "Id": "restricted-topic-policy",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789012:role/producer-role"
            },
            "Action": "sns:Publish",
            "Resource": topic_arn
        },
        {
            "Effect": "Allow",
            "Principal": {"Service": "sqs.amazonaws.com"},
            "Action": "sns:Subscribe",
            "Resource": topic_arn,
            "Condition": {
                "StringEquals": {
                    "aws:SourceAccount": "123456789012"
                }
            }
        }
    ]
}

sns.set_topic_attributes(
    TopicArn=topic_arn,
    AttributeName='Policy',
    AttributeValue=json.dumps(policy)
)

Conclusion

Scaling Amazon SNS from a prototype to a production system is fundamentally about designing the ecosystem around the service rather than the service itself. By introducing SQS buffers, implementing the claim-check pattern for large payloads, applying message filtering, enabling dead-letter queues, and establishing robust observability, you transform a simple pub/sub prototype into a resilient, cost-effective, and observable event-driven architecture. The key is to treat SNS as a high-throughput router—not a storage layer—and to push complexity like retries, ordering, and payload management into the surrounding components where you have fine-grained control. With these patterns in place, your SNS-based system can handle millions of events per day while remaining maintainable and secure.