Scaling CosmosDB: From Prototype to Production

Azure Cosmos DB is Microsoft's globally distributed, multi-model database service designed for mission-critical applications. While getting started with Cosmos DB is remarkably easy—often just a few lines of code—moving from a working prototype to a production-ready, scalable system requires careful planning around partitioning, throughput provisioning, indexing, and consistency models. This tutorial walks you through the key decisions and code patterns that separate a prototype from a production-grade Cosmos DB implementation.

Why Scaling Cosmos DB Matters

In a prototype, you typically work with small datasets, low traffic, and a single region. Cosmos DB handles these scenarios effortlessly. However, as your application grows, several challenges emerge:

Throughput costs can spiral if your partitioning strategy is poorly designed
Hot partitions can cause throttling (HTTP 429 errors) and latency spikes
Cross-region replication introduces consistency trade-offs
Indexing policies impact both query performance and RU consumption
Connection management becomes critical under high concurrency

Addressing these concerns early prevents costly refactors later. Let's explore each area with practical examples.

Understanding Request Units and Throughput

Every operation in Cosmos DB consumes Request Units (RUs). A 1 KB document read costs 1 RU. Writes are more expensive, typically starting around 5 RUs for a small document. Understanding RU consumption is the foundation of scaling.

Provisioned vs. Serverless

Cosmos DB offers two throughput models. Provisioned throughput reserves a fixed number of RUs per second, which is ideal for predictable workloads. Serverless scales automatically and charges per operation, making it suitable for sporadic or development workloads.

// Setting up a container with provisioned throughput
using Azure.Cosmos;

var cosmosClient = new CosmosClient(connectionString);
var database = await cosmosClient.CreateDatabaseIfNotExistsAsync("ProductionDB");

var containerProperties = new ContainerProperties
{
    Id = "Orders",
    PartitionKeyPath = "/customerId",
    IndexingPolicy = new IndexingPolicy
    {
        IndexingMode = IndexingMode.Consistent,
        Automatic = true,
        IncludedPaths = { new IncludedPath { Path = "/*" } },
        ExcludedPaths = { new ExcludedPath { Path = "/largeText/*" } }
    }
};

var container = await database.CreateContainerIfNotExistsAsync(
    containerProperties,
    throughput: 10000 // Manual provisioned throughput
);

For production workloads with variable traffic, consider autoscale provisioning, which scales between 10% and 100% of your configured maximum:

// Create container with autoscale throughput
var throughputProperties = ThroughputProperties.CreateAutoscaleThroughput(maxThroughput: 10000);

var container = await database.CreateContainerIfNotExistsAsync(
    containerProperties,
    throughputProperties
);

Partitioning Strategy: The Most Critical Decision

Your partition key choice determines how data is distributed across physical partitions. A poor partition key leads to hot partitions, storage imbalances, and throttling. This is the single most important architectural decision when scaling Cosmos DB.

Choosing a Good Partition Key

A good partition key has three properties: high cardinality (many distinct values), even distribution of data and traffic, and alignment with your most common query patterns. Queries that include the partition key in their filter are routed to a single partition, minimizing RU consumption.

// Example: Storing user activity events
// GOOD partition key: /userId (high cardinality, queries by user)
// BAD partition key: /eventType (low cardinality, hot partitions for popular events)

var containerProperties = new ContainerProperties
{
    Id = "UserActivity",
    PartitionKeyPath = "/userId",
    // Composite index for common query patterns
    IndexingPolicy = new IndexingPolicy
    {
        CompositeIndexes = new List<Collection<CompositePath>>
        {
            new Collection<CompositePath>
            {
                new CompositePath { Path = "/timestamp", Order = CompositePathSortOrder.Descending },
                new CompositePath { Path = "/activityType" }
            }
        }
    }
};

Hierarchical Partition Keys

For scenarios where a single partition key cannot provide both good distribution and efficient queries, Cosmos DB supports hierarchical partition keys (up to three levels). This is useful for multi-tenant applications:

// Hierarchical partitioning for multi-tenant apps
var containerProperties = new ContainerProperties("Documents")
{
    PartitionKeyPaths = new List<string> { "/tenantId", "/userId", "/documentId" }
};

// This allows:
// - Efficient queries by tenantId (broad)
// - Efficient queries by tenantId + userId (narrow)
// - Direct point reads by all three keys (most efficient)

Optimizing Queries for Scale

Point Reads vs. Queries

Point reads—fetching a single item by its partition key and item ID—are the most efficient operation in Cosmos DB, costing exactly 1 RU for a 1 KB document. Always prefer point reads when you know both the partition key and the document ID:

// INEFFICIENT: Query when you could do a point read
var query = container.GetItemQueryIterator<Order>(
    "SELECT * FROM c WHERE c.customerId = @customerId AND c.id = @orderId",
    requestOptions: new QueryRequestOptions
    {
        PartitionKey = new PartitionKey(customerId)
    }
);

// EFFICIENT: Point read (costs exactly 1 RU for 1KB)
var response = await container.ReadItemAsync<Order>(
    orderId,
    new PartitionKey(customerId)
);

Pagination and Continuation Tokens

For queries that return large result sets, always use continuation tokens to paginate results. This prevents timeouts and keeps RU consumption predictable:

string continuationToken = null;
List<Order> allOrders = new List<Order>();

do
{
    var queryRequestOptions = new QueryRequestOptions
    {
        MaxItemCount = 100, // Process 100 items at a time
        PartitionKey = new PartitionKey(customerId)
    };

    var queryResult = await container.GetItemQueryIterator<Order>(
        "SELECT * FROM c WHERE c.status = 'pending'",
        continuationToken: continuationToken,
        requestOptions: queryRequestOptions
    ).ReadNextAsync();

    allOrders.AddRange(queryResult);
    continuationToken = queryResult.ContinuationToken;

} while (continuationToken != null);

Console.WriteLine($"Retrieved {allOrders.Count} pending orders");

Bulk Execution for High-Throughput Scenarios

When ingesting large volumes of data, individual operations are inefficient. Cosmos DB's bulk executor library batches operations and handles throttling retries automatically:

// Enable bulk execution on the CosmosClient
var cosmosClient = new CosmosClient(
    connectionString,
    new CosmosClientOptions
    {
        AllowBulkExecution = true,
        MaxRequestsPerTcpConnection = 30,
        MaxTcpConnectionsPerEndpoint = 30
    }
);

var container = cosmosClient.GetContainer("ProductionDB", "Orders");

// Bulk insert thousands of documents
var tasks = new List<Task>();
var ordersToInsert = GenerateOrders(50000);

foreach (var order in ordersToInsert)
{
    tasks.Add(container.CreateItemAsync(order, new PartitionKey(order.CustomerId))
        .ContinueWith(task =>
        {
            if (task.IsFaulted)
            {
                Console.WriteLine($"Failed to insert order {order.Id}: {task.Exception?.InnerException?.Message}");
            }
        }));
}

await Task.WhenAll(tasks);
Console.WriteLine("Bulk insert completed");

Connection and Client Configuration

In production, the way you configure and manage the CosmosClient has a significant impact on performance. The client should be a singleton—created once and reused for the lifetime of the application:

// Production-ready CosmosClient configuration
public class CosmosDbService
{
    private static CosmosClient _client;
    private static readonly object _lock = new object();

    public static CosmosClient GetClient(string connectionString)
    {
        if (_client == null)
        {
            lock (_lock)
            {
                if (_client == null)
                {
                    _client = new CosmosClient(
                        connectionString,
                        new CosmosClientOptions
                        {
                            ConnectionMode = ConnectionMode.Direct,
                            ConsistencyLevel = ConsistencyLevel.Session,
                            MaxRequestsPerTcpConnection = 30,
                            MaxTcpConnectionsPerEndpoint = 30,
                            IdleTcpConnectionTimeout = TimeSpan.FromMinutes(10),
                            OpenTcpConnectionTimeout = TimeSpan.FromSeconds(5),
                            RequestTimeout = TimeSpan.FromSeconds(60),
                            RetryOptions = new CosmosClientRetryOptions
                            {
                                MaxRetryAttemptsOnRateLimitedRequests = 9,
                                MaxRetryWaitTimeOnRateLimitedRequests = TimeSpan.FromSeconds(30)
                            }
                        }
                    );
                }
            }
        }
        return _client;
    }
}

Direct mode is recommended for production because it bypasses the gateway, reducing latency. However, if your application runs in an environment with restricted networking (such as certain App Service plans), you may need to use Gateway mode.

Multi-Region Distribution and Consistency

One of Cosmos DB's most powerful features is turnkey global distribution. You can add regions to your database account with a single API call, and Cosmos DB handles data replication automatically.

Choosing a Consistency Level

Cosmos DB offers five consistency levels, each with different latency, availability, and throughput trade-offs:

Strong: Linearizable consistency, highest latency, lowest availability in multi-region setups
Bounded staleness: Consistent prefix with a configurable lag (in operations or time)
Session: Consistent within a single client session, good default for most applications
Consistent prefix: Reads never see out-of-order writes
Eventual: Lowest latency, highest availability, reads may be stale

// Configure multi-region write with preferred regions
var cosmosClient = new CosmosClient(
    connectionString,
    new CosmosClientOptions
    {
        ConsistencyLevel = ConsistencyLevel.Session,
        ApplicationPreferredRegions = new List<string>
        {
            "East US",      // Primary region
            "West Europe",  // Failover region
            "Southeast Asia" // Tertiary failover
        }
    }
);

// For multi-region write accounts, Cosmos DB automatically
// routes writes to the nearest region

Handling Conflicts in Multi-Region Writes

When using multi-region write accounts, conflicts can occur when the same document is modified in different regions simultaneously. Cosmos DB provides last-writer-wins (LWW) resolution by default, but you can implement custom conflict resolution:

// Configure custom conflict resolution with a stored procedure
var containerProperties = new ContainerProperties
{
    Id = "Inventory",
    PartitionKeyPath = "/productId",
    ConflictResolutionPolicy = new ConflictResolutionPolicy
    {
        Mode = ConflictResolutionMode.Custom,
        ConflictResolutionProcedure = "dbs/ProductionDB/colls/Inventory/sprocs/resolveConflict"
    }
};

// The stored procedure receives conflicting documents and decides which to keep

Indexing Policy Optimization

By default, Cosmos DB indexes every property in every document. While convenient for prototyping, this wastes RUs on writes and storage. In production, tailor your indexing policy to your query patterns:

// Optimized indexing policy for an e-commerce product container
var indexingPolicy = new IndexingPolicy
{
    IndexingMode = IndexingMode.Consistent,
    Automatic = true,
    IncludedPaths = { new IncludedPath { Path = "/*" } },
    ExcludedPaths =
    {
        new ExcludedPath { Path = "/description/*" },      // Large text, never queried
        new ExcludedPath { Path = "/images/*" },           // Binary data references
        new ExcludedPath { Path = "/metadata/\"_etag\"/*" } // Internal metadata
    },
    CompositeIndexes = new List<Collection<CompositePath>>
    {
        // Optimize for "get products by category, sorted by price"
        new Collection<CompositePath>
        {
            new CompositePath { Path = "/category" },
            new CompositePath { Path = "/price", Order = CompositePathSortOrder.Ascending }
        },
        // Optimize for "get products by brand, sorted by rating"
        new Collection<CompositePath>
        {
            new CompositePath { Path = "/brand" },
            new CompositePath { Path = "/rating", Order = CompositePathSortOrder.Descending }
        }
    }
};

Monitoring and Diagnostics

Production systems require observability. Cosmos DB provides detailed diagnostics through the .NET SDK that expose RU consumption, latency, and retry information:

// Capture diagnostics for every operation
var response = await container.CreateItemAsync(order, new PartitionKey(order.CustomerId));

var diagnostics = response.Diagnostics;
Console.WriteLine($"Total RU charge: {response.RequestCharge}");
Console.WriteLine($"Diagnostics: {diagnostics.ToString()}");

// In production, log these to Application Insights or your monitoring system
// Set a threshold alert if RU charge exceeds expected values

// Example: Batch query with diagnostics
var queryIterator = container.GetItemQueryIterator<Order>(
    "SELECT * FROM c WHERE c.customerId = @customerId",
    requestOptions: new QueryRequestOptions
    {
        PartitionKey = new PartitionKey(customerId),
        MaxItemCount = 50
    }
);

while (queryIterator.HasMoreResults)
{
    var response = await queryIterator.ReadNextAsync();
    _logger.LogInformation(
        "Query consumed {RUCharge} RUs, took {ElapsedMs}ms",
        response.RequestCharge,
        response.Diagnostics.GetClientElapsedTime().TotalMilliseconds
    );
    
    foreach (var order in response)
    {
        ProcessOrder(order);
    }
}

Best Practices Summary

Choose your partition key carefully: It cannot be changed after container creation. Prioritize high cardinality and even distribution.
Use point reads whenever possible: They are dramatically cheaper and faster than queries.
Right-size your throughput: Start with autoscale, monitor usage, and switch to manual provisioned if traffic becomes predictable.
Exclude large or unqueried properties from indexing: This reduces RU charges on writes and saves storage.
Reuse the CosmosClient as a singleton: Creating multiple clients exhausts connection pools and degrades performance.
Use bulk execution for batch operations: It can improve throughput by 10x or more compared to individual operations.
Monitor RU consumption continuously: Set up alerts for throttling (429 responses) and unexpected RU spikes.
Design for idempotency: Use the document ID as a natural key to make retries safe during bulk operations.
Leverage change feed for event-driven architectures: It provides a reliable, ordered log of changes for downstream processing.
Test with production-like data volumes: Partitioning and indexing issues often only manifest at scale.

Conclusion

Scaling Cosmos DB from prototype to production is less about writing more code and more about making the right architectural decisions early. Your partition key strategy, throughput model, indexing policy, and client configuration all have compounding effects as your data grows and traffic increases. By following the patterns outlined in this tutorial—choosing high-cardinality partition keys, preferring point reads, leveraging bulk execution, optimizing indexing policies, and implementing robust monitoring—you can build a Cosmos DB solution that scales smoothly from thousands to billions of documents while keeping costs predictable and performance consistent. Remember that the cheapest way to scale is to consume fewer RUs per operation, and every architectural decision should be evaluated through that lens.