S3 Security: IAM Policies and Network Security

Introduction to S3 Security

Amazon S3 is one of the most widely used cloud storage services in the world, but its flexibility also makes it a frequent target for misconfigurations. Securing S3 buckets requires a layered approach that combines identity-based controls (IAM policies) with network-level restrictions. In this tutorial, we will explore how to lock down your S3 buckets using IAM policies, bucket policies, and network security controls such as VPC endpoints and source IP restrictions.

Why S3 Security Matters

S3 buckets often store sensitive data such as application logs, user uploads, database backups, and configuration files. A single misconfigured bucket can expose millions of records to the public internet. High-profile data leaks from companies across industries have almost always traced back to one of two problems: overly permissive IAM policies or missing network restrictions. By understanding both layers, you can build defense-in-depth and reduce the blast radius of any single mistake.

Understanding the Security Layers

S3 security operates across several layers that work together:

IAM Identity-Based Policies: Attached to users, groups, or roles. They define what actions those identities can perform on which resources.
S3 Bucket Policies: Resource-based policies attached directly to a bucket. They can grant or deny access to principals, including those from other AWS accounts.
S3 Access Control Lists (ACLs): Legacy access controls. AWS now recommends disabling ACLs and using bucket policies instead.
Network Security: VPC endpoints, source IP conditions, and bucket policies that restrict access based on network origin.
Encryption: Server-side encryption with SSE-S3, SSE-KMS, or SSE-C to protect data at rest.

IAM Policies for S3

Identity-Based Policy Basics

An IAM identity-based policy is a JSON document that defines permissions for an IAM principal. When applied to a user or role, it grants access to specific S3 actions on specific resources. The key principle is least privilege: grant only the permissions needed, nothing more.

Here is a read-only IAM policy that allows a user to list and read objects from a single bucket:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ListBucketObjects",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket"
      ],
      "Resource": "arn:aws:s3:::my-secure-bucket"
    },
    {
      "Sid": "ReadBucketObjects",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject"
      ],
      "Resource": "arn:aws:s3:::my-secure-bucket/*"
    }
  ]
}

Notice the distinction between the bucket ARN (used for bucket-level actions like ListBucket) and the object ARN with the /* suffix (used for object-level actions like GetObject). This is a common source of errors for developers new to S3 policies.

Restricting Access to Specific Prefixes

You can scope permissions down to specific key prefixes within a bucket. This is useful when multiple teams share a single bucket:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ListTeamPrefix",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::shared-company-bucket",
      "Condition": {
        "StringLike": {
          "s3:prefix": [
            "team-alpha/*"
          ]
        }
      }
    },
    {
      "Sid": "ReadWriteTeamPrefix",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::shared-company-bucket/team-alpha/*"
    }
  ]
}

Denying Unencrypted Object Uploads

You can enforce encryption requirements at the IAM level by denying uploads that do not specify server-side encryption. This ensures developers cannot accidentally store plaintext data:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyUnencryptedObjectUploads",
      "Effect": "Deny",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::my-secure-bucket/*",
      "Condition": {
        "StringNotEquals": {
          "s3:x-amz-server-side-encryption": "aws:kms"
        }
      }
    }
  ]
}

S3 Bucket Policies

When to Use Bucket Policies

Bucket policies are attached to the bucket itself, not to an IAM principal. They are essential when you need to grant cross-account access, enforce organization-wide rules, or restrict access based on network conditions. A bucket policy can also deny access even when an IAM policy allows it, making it a powerful enforcement tool.

Restricting Access to a VPC Endpoint

One of the most effective network security controls is requiring all S3 access to come through a specific VPC endpoint. This prevents access from the public internet while still allowing EC2 instances and Lambda functions in your VPC to reach the bucket. The following bucket policy denies any request that does not originate from the specified VPC endpoint:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyAccessOutsideVPCe",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::my-private-bucket",
        "arn:aws:s3:::my-private-bucket/*"
      ],
      "Condition": {
        "StringNotEquals": {
          "aws:sourceVpce": "vpce-1a2b3c4d5e6f7g8h9"
        }
      }
    }
  ]
}

Be careful when applying this type of policy. If the VPC endpoint ID is incorrect, you will lock yourself out of the bucket. Always test with a non-production bucket first.

Restricting Access by Source IP

If you do not use a VPC endpoint, you can still restrict access to specific public IP addresses or CIDR blocks. This is useful for allowing access only from your corporate office network:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyAccessOutsideCorporateIP",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::my-corporate-bucket",
        "arn:aws:s3:::my-corporate-bucket/*"
      ],
      "Condition": {
        "NotIpAddress": {
          "aws:SourceIp": [
            "203.0.113.0/24",
            "198.51.100.10"
          ]
        }
      }
    }
  ]
}

Enforcing HTTPS-Only Access

To prevent plaintext HTTP requests, you can deny any request that uses aws:SecureTransport set to false. This forces all clients to use TLS:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyInsecureTransport",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::my-secure-bucket",
        "arn:aws:s3:::my-secure-bucket/*"
      ],
      "Condition": {
        "Bool": {
          "aws:SecureTransport": "false"
        }
      }
    }
  ]
}

Network Security with VPC Endpoints

Gateway vs. Interface Endpoints

AWS provides two types of VPC endpoints for S3. Gateway endpoints are free, highly scalable, and route traffic through the AWS network without traversing the public internet. They are configured as route table entries and are the recommended choice for most S3 workloads. Interface endpoints use PrivateLink and incur hourly charges, but they support on-premises access through VPN or Direct Connect.

Creating a Gateway Endpoint with Terraform

The following Terraform configuration creates a VPC, a private subnet, and a gateway endpoint for S3. The endpoint policy restricts access to a single bucket:

resource "aws_vpc" "main" {
  cidr_block           = "10.0.0.0/16"
  enable_dns_support   = true
  enable_dns_hostnames = true
}

resource "aws_subnet" "private" {
  vpc_id     = aws_vpc.main.id
  cidr_block = "10.0.1.0/24"
}

resource "aws_vpc_endpoint" "s3" {
  vpc_id            = aws_vpc.main.id
  service_name      = "com.amazonaws.us-east-1.s3"
  vpc_endpoint_type = "Gateway"
  route_table_ids   = [aws_vpc.main.default_route_table_id]

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect    = "Allow"
        Principal = "*"
        Action    = ["s3:GetObject", "s3:ListBucket"]
        Resource = [
          "arn:aws:s3:::my-private-bucket",
          "arn:aws:s3:::my-private-bucket/*"
        ]
      }
    ]
  })
}

Endpoint Policies vs. Bucket Policies

Both endpoint policies and bucket policies can restrict S3 access, but they operate at different points in the request path. An endpoint policy is evaluated when traffic passes through the VPC endpoint, while a bucket policy is evaluated when the request reaches S3. For maximum security, use both: the endpoint policy limits what can leave your VPC, and the bucket policy limits what can reach the bucket. This layered approach means that even if one policy is misconfigured, the other still provides protection.

Blocking Public Access

AWS provides S3 Block Public Access as a safety net that overrides any policy that might accidentally grant public access. You should enable all four settings at both the bucket level and the account level:

BlockPublicAcls — blocks new public ACLs and ignores existing ones.
IgnorePublicAcls — treats existing public ACLs as if they do not exist.
BlockPublicPolicy — rejects bucket policies that grant public access.
RestrictPublicBuckets — restricts access to buckets with public policies to authenticated users only.

Here is how to enable Block Public Access using the AWS CLI:

aws s3api put-public-access-block \
  --bucket my-secure-bucket \
  --public-access-block-configuration \
  BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true

Best Practices

Apply least privilege: Grant only the specific actions and resources needed. Avoid using s3:* or Resource: "*" unless absolutely necessary.
Use bucket policies for cross-account access: Do not create IAM users in your account for external parties. Instead, grant access to their IAM role ARN via a bucket policy.
Enable Block Public Access by default: Turn it on at the account level so that new buckets are protected automatically.
Enforce encryption: Use bucket policies or IAM policies to deny unencrypted uploads. Prefer SSE-KMS when you need customer-managed keys and audit trails.
Use VPC endpoints for private workloads: Route S3 traffic through gateway endpoints and deny direct internet access with bucket policies.
Enable access logging and CloudTrail: S3 server access logs and AWS CloudTrail data events provide an audit trail for every API call against your buckets.
Rotate credentials and use roles: Prefer IAM roles over long-lived access keys. Use IAM Roles for Service Accounts (IRSA) on EKS and instance profiles on EC2.
Tag your buckets: Use tags to classify data sensitivity and feed them into AWS Config rules or SCPs for automated compliance checks.
Test policies with the IAM Policy Simulator: Before deploying, validate that your policies produce the expected allow or deny decisions for representative requests.

Putting It All Together

The following bucket policy combines several controls into a single document. It denies public access, requires HTTPS, requires requests to come through a specific VPC endpoint, and enforces KMS encryption on uploads:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyInsecureTransport",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::production-data-bucket",
        "arn:aws:s3:::production-data-bucket/*"
      ],
      "Condition": {
        "Bool": {
          "aws:SecureTransport": "false"
        }
      }
    },
    {
      "Sid": "DenyAccessOutsideVPCe",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::production-data-bucket",
        "arn:aws:s3:::production-data-bucket/*"
      ],
      "Condition": {
        "StringNotEquals": {
          "aws:sourceVpce": "vpce-1a2b3c4d5e6f7g8h9"
        }
      }
    },
    {
      "Sid": "DenyUnencryptedUploads",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::production-data-bucket/*",
      "Condition": {
        "StringNotEquals": {
          "s3:x-amz-server-side-encryption": "aws:kms"
        }
      }
    }
  ]
}

Conclusion

Securing S3 is not a single configuration step but a combination of identity-based controls, resource-based policies, and network restrictions working together. By applying least-privilege IAM policies, enforcing encryption and HTTPS through bucket policies, routing traffic through VPC endpoints, and enabling Block Public Access, you create multiple layers of defense that protect your data even if one layer fails. Start with the strictest reasonable configuration, test thoroughly in a non-production environment, and continuously monitor with CloudTrail and AWS Config to catch drift before it becomes a breach. With these practices in place, your S3 buckets will be resilient against both accidental misconfiguration and targeted attacks.