← Back to DevBytes
Developer Tutorial August 04, 2026 7 min read

Scaling Spanner: From Prototype to Production

Introduction to Scaling Spanner

Google Cloud Spanner is a globally distributed, strongly consistent relational database that combines the scalability of NoSQL systems with the familiar semantics of a SQL database. While building a prototype on Spanner is straightforward—create an instance, define a schema, and start inserting rows—moving that prototype to a production-grade system that handles real traffic requires careful planning around schema design, data distribution, transaction patterns, and operational monitoring.

This tutorial walks through the journey of taking a Spanner-backed application from a working prototype to a robust production deployment. We will cover schema evolution strategies, choosing the right primary keys, managing interleaved tables, optimizing queries, handling transactions, configuring instances for high availability, and monitoring performance.

Why Scaling Spanner Matters

Spanner's promise is horizontal scalability without sacrificing ACID guarantees. However, that promise only materializes when your schema and access patterns align with how Spanner distributes data. A poorly designed schema can turn a system capable of millions of reads per second into one that bottlenecks on a single split, effectively serializing all your traffic through one node.

The key insight is that Spanner shards data into "splits," each of which is a range of rows in a table ordered by primary key. If your hot rows are clustered together, they end up on the same split, and that split becomes a bottleneck. Scaling Spanner is largely about ensuring work spreads evenly across splits and that transactions avoid touching too many splits unnecessarily.

From Prototype to Production: The Journey

1. Designing Scalable Primary Keys

The most common mistake in Spanner prototypes is using a monotonically increasing primary key, such as a timestamp or an auto-incrementing integer. Because Spanner orders rows by primary key, all new inserts land at the end of the key range, concentrating writes on a single split. This creates hotspots that limit throughput.

Instead, use techniques that distribute writes evenly:

Here is an example of a sharded primary key approach for an orders table:

-- Prototype version (hotspot-prone)
CREATE TABLE Orders (
  order_id   INT64 NOT NULL,
  customer_id INT64 NOT NULL,
  total      NUMERIC,
  created_at TIMESTAMP,
) PRIMARY KEY (order_id);

-- Production version (sharded)
CREATE TABLE Orders (
  shard_id   INT64 NOT NULL,
  order_id   INT64 NOT NULL,
  customer_id INT64 NOT NULL,
  total      NUMERIC,
  created_at TIMESTAMP,
) PRIMARY KEY (shard_id, order_id);

When inserting, compute the shard ID by hashing the order ID modulo the number of shards:

const shardCount = 64;

function shardId(orderId) {
  // Use a hash to distribute evenly
  const hash = require('crypto').createHash('md5')
    .update(String(orderId)).digest();
  return hash.readUInt32BE(0) % shardCount;
}

async function insertOrder(spanner, orderId, customerId, total) {
  const sid = shardId(orderId);
  await spanner.database.runTransactionAsync(async (tx) => {
    await tx.insert('Orders', {
      shard_id: sid,
      order_id: orderId,
      customer_id: customerId,
      total: total,
      created_at: Spanner.timestamp(),
    });
    await tx.commit();
  });
}

2. Using Interleaved Tables for Co-located Data

Spanner supports interleaved tables, where a child table's rows are physically stored alongside their parent rows. This is powerful for one-to-many relationships that are frequently accessed together, because it allows a single split to serve queries spanning both tables without cross-node coordination.

CREATE TABLE Customers (
  customer_id INT64 NOT NULL,
  name        STRING(100),
  email       STRING(200),
) PRIMARY KEY (customer_id);

CREATE TABLE Orders (
  customer_id INT64 NOT NULL,
  order_id    INT64 NOT NULL,
  total       NUMERIC,
  created_at  TIMESTAMP,
) PRIMARY KEY (customer_id, order_id),
  INTERLEAVE IN PARENT Customers ON DELETE CASCADE;

With this design, all orders for a given customer are co-located with that customer's row. A query fetching a customer and their recent orders touches a single split. However, be cautious: if a single parent has an unbounded number of children (for example, a popular product with millions of reviews), the interleaved split can grow too large to split efficiently. Interleaving works best when the number of children per parent is bounded and modest.

3. Schema Evolution Without Downtime

In production, you cannot afford to lock tables while altering schemas. Spanner supports online schema changes, but large tables can take time to backfill. Follow these practices:

-- Step 1: Add nullable column (fast, no backfill needed)
ALTER TABLE Orders ADD COLUMN priority INT64;

-- Step 2: Backfill in batches from application code
-- Step 3: Create index asynchronously
CREATE INDEX Idx_Orders_Priority
  ON Orders (priority)
  ASYNC;

4. Optimizing Queries for Production

Spanner's query optimizer is powerful, but it relies on statistics and indexes. In production, always verify query plans using the EXPLAIN or EXPLAIN ANALYZE statements. Look for full scans, which indicate missing indexes, and for high "rows scanned" counts relative to "rows returned."

EXPLAIN ANALYZE
SELECT o.order_id, o.total, c.name
FROM Orders o
JOIN Customers c ON o.customer_id = c.customer_id
WHERE o.created_at >= TIMESTAMP '2024-01-01'
  AND o.total > 100;

Common optimizations include:

CREATE INDEX Idx_Orders_Customer_Total
  ON Orders (customer_id, total DESC)
  STORING (order_id, created_at);

5. Transaction Patterns and Contention

Spanner provides read-write transactions with strict serializability. However, long-running transactions or transactions that touch many rows can cause contention and lock timeouts. In production, keep transactions short and scoped.

For read-heavy workloads where stale reads are acceptable, use stale reads with a timestamp bound. This reduces load on the leader and improves throughput:

const [rows] = await database.run({
  sql: 'SELECT order_id, total FROM Orders WHERE customer_id = @cid',
  params: { cid: customerId },
  // Read data that is at most 15 seconds old
  timestampBound: Spanner.timestampBound(
    Spanner.exactStaleness(15)
  ),
});

For write-heavy workloads, batch mutations and commit them in a single transaction rather than committing row by row:

await database.runTransactionAsync(async (tx) => {
  const mutations = orders.map((o) => ({
    table: 'Orders',
    columns: ['shard_id', 'order_id', 'customer_id', 'total', 'created_at'],
    values: [[o.shardId, o.orderId, o.customerId, o.total, o.createdAt]],
  }));
  await tx.batchInsertOrUpdate(mutations);
  await tx.commit();
});

6. Configuring Instances for High Availability

For production, choose a multi-region configuration if your application requires global availability and low-latency reads across regions. Single-region configurations are cheaper but do not survive a regional outage.

gcloud spanner instances create prod-instance \
  --config=nam-eur-asia1 \
  --description="Production Spanner Instance" \
  --nodes=3

Key considerations:

7. Monitoring and Observability

Production readiness requires visibility into key metrics. Use Cloud Monitoring to track:

gcloud monitoring dashboards create \
  --config-from-file=spanner-dashboard.json

Set up alerting policies for critical thresholds:

gcloud alpha monitoring policies create \
  --policy-from-file=spanner-alerts.yaml

Best Practices Summary

Conclusion

Scaling Spanner from a prototype to a production system is less about brute-force provisioning and more about aligning your schema and access patterns with Spanner's distributed architecture. By choosing distribution-friendly primary keys, leveraging interleaved tables judiciously, evolving schemas without downtime, optimizing queries with covering indexes, keeping transactions lean, configuring instances for high availability, and maintaining rigorous observability, you can unlock Spanner's full potential: a globally consistent database that scales linearly with your workload. The investment in thoughtful design up front pays dividends in reliability, performance, and operational simplicity as your application grows.

— Ad —

Google AdSense will appear here after approval

← Back to all articles