Understanding Video Streaming Architecture with Apache Cassandra
Video streaming platforms demand a database architecture capable of handling massive write throughput, low-latency reads, and seamless horizontal scalability. Apache Cassandra, a distributed NoSQL database, is purpose-built for these exact requirements. In this tutorial, you will learn how to design a robust video streaming data layer using Cassandra, covering data modeling, practical implementation, and production-tested best practices.
What It Is
Designing a video streaming system with Cassandra means structuring your data to efficiently store and retrieve video metadata, user profiles, watch history, recommendations, and playback analytics across a globally distributed cluster. Unlike traditional relational databases, Cassandra uses a denormalized, query-first data model optimized for specific access patterns. Video metadata—such as titles, descriptions, thumbnails, encoding formats, and DRM keys—lives alongside user interaction data in tables designed to serve reads with minimal latency, even under extreme concurrency.
The core insight is that Cassandra treats video streaming data as time-series events and entity snapshots, leveraging its log-structured merge-tree storage engine to absorb millions of writes per second while maintaining predictable read performance through partition key-based data locality.
Why Cassandra Matters for Video Streaming
Modern video platforms face several non-negotiable demands that make Cassandra an ideal fit:
- Massive write throughput: Every play, pause, seek, and quality change event generates a write. At scale, this translates to hundreds of thousands of writes per second. Cassandra's append-only write path and lack of locking mechanisms allow it to ingest these workloads effortlessly.
- Global distribution: Video content is consumed worldwide. Cassandra's multi-datacenter replication with configurable consistency levels (LOCAL_QUORUM, EACH_QUORUM) lets you serve reads from the closest datacenter while synchronizing writes asynchronously across regions.
- Predictable low latency: Cassandra's partition-per-hash design ensures that reads touch a bounded number of SSTables. For a well-modeled partition, a read typically accesses one or two SSTables, delivering sub-millisecond p99 latency when using SSDs.
- Linear scalability: Adding nodes to a Cassandra cluster increases capacity and throughput linearly without downtime or complex rebalancing operations. This aligns perfectly with the growth trajectory of a successful streaming service.
- Fault tolerance: With no single point of failure and tunable replication factors, Cassandra continues serving reads and writes even when entire nodes or datacenters go offline.
Core Data Modeling Concepts for Video Streaming
Cassandra data modeling requires a paradigm shift from normalization to denormalization. You design tables around your application's specific queries, not around the entities themselves. For a video streaming platform, typical query patterns include:
- Retrieving video details by video ID
- Fetching a user's watch history, paginated by timestamp
- Loading a playlist's videos in sequence order
- Aggregating view counts per video within a time window
- Looking up recommendations for a user based on genre preferences
Each of these patterns demands its own dedicated table, often with a partition key that groups related data together for efficient retrieval.
Practical Implementation: Step-by-Step Code Examples
Let's build the data layer for a video streaming service called "StreamVault." We'll create tables, write data, and execute the queries that power the platform. All examples use the Cassandra Query Language (CQL) and the DataStax Java Driver for application code.
1. Setting Up the Keyspace and Replication Strategy
First, create a keyspace with NetworkTopologyStrategy to enable multi-datacenter replication. This is essential for a globally distributed streaming service.
-- Create keyspace for StreamVault
CREATE KEYSPACE streamvault
WITH replication = {
'class': 'NetworkTopologyStrategy',
'us-east': 3,
'eu-west': 3,
'ap-southeast': 2
}
AND durable_writes = true;
-- Switch to the keyspace
USE streamvault;
The replication factor of 3 in us-east and eu-west means each write is stored on three nodes in those datacenters, providing strong durability and local read availability. The ap-southeast datacenter uses a factor of 2 for lighter regional coverage.
2. The Videos Table: Storing Core Video Metadata
The videos table stores essential metadata for each piece of content. The partition key is video_id, and we include columns for encoding information, DRM details, and thumbnail references.
CREATE TABLE videos (
video_id UUID,
title TEXT,
description TEXT,
duration_seconds INT,
thumbnail_url TEXT,
content_rating TEXT,
genre TEXT,
release_date TIMESTAMP,
encoding_profiles MAP, -- e.g., 'h264_1080p': 's3://bucket/path'
drm_key_id TEXT,
created_at TIMESTAMP,
PRIMARY KEY (video_id)
);
-- Insert a sample video
INSERT INTO videos (
video_id, title, description, duration_seconds,
thumbnail_url, content_rating, genre, release_date,
encoding_profiles, drm_key_id, created_at
) VALUES (
3b041d8b-7c6b-4c1a-9e0e-5a7f3d2b1c4e,
'The Hidden Algorithm',
'A documentary exploring machine learning origins.',
5820,
'https://cdn.streamvault.com/thumbs/hidden-algorithm.jpg',
'PG-13',
'Documentary',
'2024-09-15',
{'h264_360p': 's3://streamvault-media/hidden-algo/360p/playlist.m3u8',
'h264_1080p': 's3://streamvault-media/hidden-algo/1080p/playlist.m3u8',
'h265_2160p': 's3://streamvault-media/hidden-algo/4k/playlist.m3u8'},
'drm-7a9f3b2c-4e5d',
'2024-09-01T08:00:00'
);
The encoding_profiles column uses a map collection type to store multiple encoding variants and their manifest URLs. This allows the playback client to select the appropriate quality based on bandwidth conditions. The drm_key_id references the key management system for content decryption.
3. The Watch History Table: Tracking User Playback Events
Watch history is a time-series pattern. We partition by user_id and cluster by watched_at in descending order to efficiently retrieve the most recent watches. Each row represents a single playback session with progress tracking.
CREATE TABLE watch_history (
user_id UUID,
watched_at TIMESTAMP,
video_id UUID,
progress_seconds INT,
completed BOOLEAN,
device_type TEXT,
ip_address INET,
quality_played TEXT,
session_id UUID,
PRIMARY KEY (user_id, watched_at, video_id)
) WITH CLUSTERING ORDER BY (watched_at DESC, video_id ASC);
-- Record a watch event
INSERT INTO watch_history (
user_id, watched_at, video_id,
progress_seconds, completed, device_type,
ip_address, quality_played, session_id
) VALUES (
9f2a1b3c-4d5e-6f7a-8b9c-0d1e2f3a4b5c,
toTimestamp(now()),
3b041d8b-7c6b-4c1a-9e0e-5a7f3d2b1c4e,
1245,
false,
'AndroidTV',
'192.168.1.42',
'h264_1080p',
1a2b3c4d-5e6f-7a8b-9c0d-e1f2a3b4c5d6
);
The clustering order of watched_at DESC ensures that the most recent entries appear first when querying. The video_id as an additional clustering column allows multiple watches of the same video at different times to coexist without overwriting each other.
To retrieve the last 20 watched videos for a user:
SELECT video_id, watched_at, progress_seconds, completed
FROM watch_history
WHERE user_id = 9f2a1b3c-4d5e-6f7a-8b9c-0d1e2f3a4b5c
ORDER BY watched_at DESC
LIMIT 20;
4. The Playlists Table: Managing Curated Collections
Playlists require ordered video sequences. We use playlist_id as the partition key and position as the clustering column to maintain video order.
CREATE TABLE playlists (
playlist_id UUID,
position INT,
video_id UUID,
added_at TIMESTAMP,
added_by TEXT,
PRIMARY KEY (playlist_id, position)
) WITH CLUSTERING ORDER BY (position ASC);
-- Populate a playlist with multiple videos
BEGIN BATCH
INSERT INTO playlists (playlist_id, position, video_id, added_at, added_by)
VALUES (a1b2c3d4-e5f6-7a8b-9c0d-1e2f3a4b5c6d, 0,
3b041d8b-7c6b-4c1a-9e0e-5a7f3d2b1c4e, toTimestamp(now()), 'curator_bot');
INSERT INTO playlists (playlist_id, position, video_id, added_at, added_by)
VALUES (a1b2c3d4-e5f6-7a8b-9c0d-1e2f3a4b5c6d, 1,
7c8d9e0f-1a2b-3c4d-5e6f-7a8b9c0d1e2f, toTimestamp(now()), 'curator_bot');
INSERT INTO playlists (playlist_id, position, video_id, added_at, added_by)
VALUES (a1b2c3d4-e5f6-7a8b-9c0d-1e2f3a4b5c6d, 2,
4d5e6f7a-8b9c-0d1e-2f3a-4b5c6d7e8f9a, toTimestamp(now()), 'curator_bot');
APPLY BATCH;
Note: Batches in Cassandra should be used judiciously. In this case, all inserts share the same partition (playlist_id), making it a single-partition batch that is safe and efficient. Avoid multi-partition batches for performance-sensitive operations.
Query a playlist in order:
SELECT position, video_id
FROM playlists
WHERE playlist_id = a1b2c3d4-e5f6-7a8b-9c0d-1e2f3a4b5c6d
ORDER BY position ASC;
5. The View Counts Table: Real-Time Analytics with Counters
Cassandra provides a native counter type for distributed counting operations. This table tracks view counts per video, partitioned by date to enable time-bucketed queries and prevent counter hotspots.
CREATE TABLE video_view_counts (
video_id UUID,
view_date DATE,
total_views COUNTER,
unique_viewers COUNTER,
total_watch_minutes COUNTER,
PRIMARY KEY (video_id, view_date)
) WITH CLUSTERING ORDER BY (view_date DESC);
-- Increment view counters for today's date
UPDATE video_view_counts
SET total_views = total_views + 1,
unique_viewers = unique_viewers + 1,
total_watch_minutes = total_watch_minutes + 97
WHERE video_id = 3b041d8b-7c6b-4c1a-9e0e-5a7f3d2b1c4e
AND view_date = '2025-04-04';
-- Query view counts for the last 7 days
SELECT view_date, total_views, unique_viewers, total_watch_minutes
FROM video_view_counts
WHERE video_id = 3b041d8b-7c6b-4c1a-9e0e-5a7f3d2b1c4e
AND view_date >= '2025-03-28'
ORDER BY view_date DESC;
Counters are eventually consistent and should never be used in batches or logged batches. Each counter update is idempotent from the client's perspective, but the actual value converges through Cassandra's anti-entropy repair mechanisms.
6. User Recommendations: Pre-Computed Suggestions Table
Recommendations are pre-computed offline by a machine learning pipeline and stored in Cassandra for fast retrieval. The table partitions by user ID and clusters by a relevance score.
CREATE TABLE user_recommendations (
user_id UUID,
relevance_score DECIMAL,
video_id UUID,
reason TEXT,
generated_at TIMESTAMP,
PRIMARY KEY (user_id, relevance_score, video_id)
) WITH CLUSTERING ORDER BY (relevance_score DESC, video_id ASC);
-- Insert pre-computed recommendations
INSERT INTO user_recommendations (
user_id, relevance_score, video_id, reason, generated_at
) VALUES (
9f2a1b3c-4d5e-6f7a-8b9c-0d1e2f3a4b5c,
0.98,
3b041d8b-7c6b-4c1a-9e0e-5a7f3d2b1c4e,
'Based on your interest in AI documentaries',
toTimestamp(now())
);
INSERT INTO user_recommendations (
user_id, relevance_score, video_id, reason, generated_at
) VALUES (
9f2a1b3c-4d5e-6f7a-8b9c-0d1e2f3a4b5c,
0.87,
7c8d9e0f-1a2b-3c4d-5e6f-7a8b9c0d1e2f,
'Popular among viewers who watched The Hidden Algorithm',
toTimestamp(now())
);
-- Fetch top 10 recommendations for a user
SELECT video_id, relevance_score, reason
FROM user_recommendations
WHERE user_id = 9f2a1b3c-4d5e-6f7a-8b9c-0d1e2f3a4b5c
ORDER BY relevance_score DESC
LIMIT 10;
7. Application Integration: Java Driver Example
Here's how a streaming backend service queries Cassandra using the DataStax Java driver with proper connection pooling and asynchronous operations:
// Java code using the DataStax driver (version 4.x)
import com.datastax.oss.driver.api.core.CqlSession;
import com.datastax.oss.driver.api.core.cql.AsyncResultSet;
import com.datastax.oss.driver.api.core.cql.PreparedStatement;
import com.datastax.oss.driver.api.core.cql.Row;
import java.util.UUID;
import java.util.concurrent.CompletionStage;
public class StreamVaultService {
private final CqlSession session;
private final PreparedStatement getVideoStmt;
private final PreparedStatement getWatchHistoryStmt;
private final PreparedStatement getRecommendationsStmt;
public StreamVaultService() {
this.session = CqlSession.builder()
.withKeyspace("streamvault")
.withLocalDatacenter("us-east")
.build();
this.getVideoStmt = session.prepare(
"SELECT title, description, duration_seconds, encoding_profiles, drm_key_id " +
"FROM videos WHERE video_id = ?");
this.getWatchHistoryStmt = session.prepare(
"SELECT video_id, watched_at, progress_seconds, completed " +
"FROM watch_history WHERE user_id = ? ORDER BY watched_at DESC LIMIT ?");
this.getRecommendationsStmt = session.prepare(
"SELECT video_id, relevance_score, reason " +
"FROM user_recommendations WHERE user_id = ? " +
"ORDER BY relevance_score DESC LIMIT ?");
}
public CompletionStage fetchVideo(UUID videoId) {
return session.executeAsync(getVideoStmt.bind(videoId))
.thenApply(resultSet -> {
Row row = resultSet.one();
if (row == null) return null;
return new VideoMetadata(
row.getString("title"),
row.getString("description"),
row.getInt("duration_seconds"),
row.getMap("encoding_profiles", String.class, String.class),
row.getString("drm_key_id")
);
});
}
public CompletionStage> fetchWatchHistory(
UUID userId, int limit) {
return session.executeAsync(
getWatchHistoryStmt.bind(userId, limit))
.thenApply(resultSet -> {
List entries = new ArrayList<>();
for (Row row : resultSet) {
entries.add(new WatchEntry(
row.getUuid("video_id"),
row.getInstant("watched_at"),
row.getInt("progress_seconds"),
row.getBoolean("completed")
));
}
return entries;
});
}
public CompletionStage> fetchRecommendations(
UUID userId, int limit) {
return session.executeAsync(
getRecommendationsStmt.bind(userId, limit))
.thenApply(resultSet -> {
List recs = new ArrayList<>();
for (Row row : resultSet) {
recs.add(new Recommendation(
row.getUuid("video_id"),
row.getBigDecimal("relevance_score").doubleValue(),
row.getString("reason")
));
}
return recs;
});
}
public void close() {
if (session != null) session.close();
}
}
This service class demonstrates prepared statements for performance, async execution for non-blocking I/O, and proper resource management. Prepared statements are cached on the server side and reduce parsing overhead for frequently executed queries.
8. Handling Concurrent Writes and Conflict Resolution
In a distributed streaming platform, multiple services may update the same data concurrently. Cassandra resolves conflicts using last-write-wins semantics based on write timestamps. For use cases requiring more sophisticated conflict resolution, use the writetime function and collection operations:
-- Safely append to a user's list of favorited genres using a set collection
CREATE TABLE user_preferences (
user_id UUID PRIMARY KEY,
favorited_genres SET,
autoplay_enabled BOOLEAN,
subtitle_language TEXT
);
-- Add a genre to the set (idempotent operation)
UPDATE user_preferences
SET favorited_genres = favorited_genres + {'Sci-Fi'}
WHERE user_id = 9f2a1b3c-4d5e-6f7a-8b9c-0d1e2f3a4b5c;
-- Remove a genre
UPDATE user_preferences
SET favorited_genres = favorited_genres - {'Romance'}
WHERE user_id = 9f2a1b3c-4d5e-6f7a-8b9c-0d1e2f3a4b5c;
-- Query the current preferences
SELECT favorited_genres, autoplay_enabled, subtitle_language
FROM user_preferences
WHERE user_id = 9f2a1b3c-4d5e-6f7a-8b9c-0d1e2f3a4b5c;
Set operations like addition and removal are safe for concurrent updates because Cassandra applies them as delta mutations rather than full overwrites.
Best Practices for Production Video Streaming with Cassandra
Running Cassandra at scale for a video streaming workload requires adherence to several critical practices. The following guidelines are distilled from real-world deployments handling billions of events daily.
- Partition size matters: Keep partition sizes under 100 MB. For time-series data like watch history, consider bucketing by month or using a composite partition key (user_id + month_bucket) to prevent unbounded partition growth. An excessively large partition causes heap pressure during compaction and uneven load distribution across nodes.
- Choose consistency levels strategically: For user-facing reads that demand the freshest data (like checking watch progress), use
LOCAL_QUORUMto ensure strong consistency within the local datacenter. For analytics queries where slight staleness is acceptable,LOCAL_ONEprovides lower latency and higher availability. Write consistency should typically match read requirements—LOCAL_QUORUMwrites paired withLOCAL_QUORUMreads guarantee observed consistency. - Leverage TTL for data lifecycle management: Raw playback events don't need to persist indefinitely. Set a time-to-live (TTL) on analytics data to automatically expire old records and reduce storage costs.
-- Insert a playback event that auto-expires after 90 days
INSERT INTO watch_history (
user_id, watched_at, video_id, progress_seconds,
completed, device_type, session_id
) VALUES (
9f2a1b3c-4d5e-6f7a-8b9c-0d1e2f3a4b5c,
toTimestamp(now()),
3b041d8b-7c6b-4c1a-9e0e-5a7f3d2b1c4e,
2340, false, 'iOS',
8f9a0b1c-2d3e-4f5a-6b7c-8d9e0f1a2b3c
) USING TTL 7776000;
TimeWindowCompactionStrategy (TWCS), which groups SSTables by write time windows and compacts only within those windows. Tables with frequent updates, like user_preferences, work better with LeveledCompactionStrategy (LCS) to minimize read amplification.-- Alter watch_history to use TimeWindowCompactionStrategy
ALTER TABLE watch_history
WITH compaction = {
'class': 'TimeWindowCompactionStrategy',
'compaction_window_unit': 'DAYS',
'compaction_window_size': 1
};
LIMIT clause and leverage clustering key range filters. For pagination through watch history, use the watched_at timestamp from the last retrieved row as the starting point for the next page rather than relying on offset-based pagination, which becomes exponentially slower.-inc flag for incremental repair.Conclusion
Designing a video streaming data layer with Apache Cassandra is an exercise in query-first thinking, denormalized schema design, and distributed systems pragmatism. By modeling tables around precise access patterns—videos by ID, watch history by user, playlists by position, and recommendations by relevance score—you create a system that serves millions of concurrent users with consistent low latency. Cassandra's native counters, collection types, TTL support, and tunable consistency levels provide the tools needed to handle everything from real-time view counting to geo-distributed content delivery. The best practices outlined here—partition size management, compaction strategy selection, token-aware routing, and disciplined batching—form the foundation of a production-grade deployment. As your streaming platform grows, Cassandra scales linearly alongside it, ensuring that your data infrastructure never becomes the bottleneck between a user and their next favorite video.