← Back to DevBytes
Developer Tutorial July 16, 2026 14 min read

MessagePack Format: A Complete Reference Guide

What Is MessagePack

MessagePack is a binary serialization format that lets you exchange structured data between applications in a compact, efficient way. Think of it as "JSON on steroids" — it preserves the same flexible, schema-less data model you know from JSON (numbers, strings, arrays, maps, booleans, null), but encodes everything in binary rather than plain text.

The result is dramatically smaller payloads. A JSON document that weighs 1 KB might occupy just 600–700 bytes in MessagePack, with no loss of semantic information. Because the format is binary, parsing is also significantly faster — there are no quotes to strip, no whitespace to skip, and numeric values are stored in their native binary representation instead of being converted to and from decimal strings.

MessagePack was originally created by Sadayuki Furuhashi and is now governed by the msgpack.org community. It has official implementations in over 50 languages, making it one of the most universally supported binary serialization formats available today.

Key Characteristics at a Glance

Why MessagePack Matters

🚀 Deploy your AI agent in 10 minutes

Managed Hermes hosting. Zero DevOps. 100M tokens/mo included.

Try it free →

Modern applications frequently exchange data across network boundaries, write to on-disk caches, or communicate between microservices. In all these scenarios, serialization overhead directly impacts latency, bandwidth costs, and CPU utilization. MessagePack addresses these concerns in ways that alternatives often cannot match.

MessagePack vs JSON

JSON is ubiquitous and human-readable, but it has inherent inefficiencies. Consider the following JSON object:

{"compact":true,"schema":0,"id":42,"tags":["fast","binary"]}

That is 60 bytes. The equivalent MessagePack encoding is 42 bytes — a 30% reduction. The savings come from:

For large arrays of floating-point numbers, the difference is even more pronounced. JSON must convert each number to a decimal string, while MessagePack stores IEEE 754 doubles in exactly 9 bytes (1 tag byte + 8 data bytes), regardless of the number's magnitude.

MessagePack vs Protocol Buffers

Protocol Buffers (protobuf) are even more compact than MessagePack in many cases because they use field numbers instead of field names. However, protobuf requires a schema — you must define your data structures in .proto files and run a code generator before you can serialize anything. MessagePack requires zero preparation: serialize any object, anywhere, at any time. This makes MessagePack ideal for:

Performance Benchmarks

Independent benchmarks consistently show MessagePack parsing at 2×–5× the speed of JSON parsing. For example, in Python using the official msgpack library (which is backed by a C extension), deserializing a 1 MB payload typically completes in under 10 milliseconds, whereas the equivalent json.loads() call might take 30–50 ms. In high-throughput systems processing millions of messages per second, this gap translates directly to reduced hardware requirements.

Format Specification Deep Dive

Understanding the wire format helps you debug encoding issues and appreciate how MessagePack achieves its compactness. Every value on the wire begins with a single tag byte that communicates both the type and, for short values, the data itself.

Tag Byte Overview

The first byte of any encoded value falls into one of these ranges:

0x00 – 0x7f : positive fixint (stores the integer directly in the tag)
0x80 – 0x8f : fixmap (map with up to 15 key-value pairs)
0x90 – 0x9f : fixarray (array with up to 15 elements)
0xa0 – 0xbf : fixstr (string with up to 31 bytes)
0xc0 – 0xdf : miscellaneous (nil, false, true, float, extended types)
0xe0 – 0xff : negative fixint (stores -1 to -32 directly in the tag)

This design is brilliant in its simplicity. Values that occur frequently — small integers, short strings, tiny collections — are encoded in a single byte with zero overhead.

Integers

MessagePack supports signed integers up to 64 bits. The encoding uses the smallest representation that fits the value:

Positive fixint: 0xxxxxxx (0x00–0x7f, stores 0 to 127)
Negative fixint: 111yyyyy (0xe0–0xff, stores -32 to -1; note that 0xff is -1)
uint 8: 0xcc followed by 1 byte (0 to 255)
uint 16: 0xcd followed by 2 bytes big-endian (0 to 65535)
uint 32: 0xce followed by 4 bytes big-endian (0 to 4294967295)
uint 64: 0xcf followed by 8 bytes big-endian
int 8: 0xd0 followed by 1 byte (-128 to 127)
int 16: 0xd1 followed by 2 bytes big-endian
int 32: 0xd2 followed by 4 bytes big-endian
int 64: 0xd3 followed by 8 bytes big-endian

Example encodings:

42          → 0x2a                          (fixint, 1 byte)
1000        → 0xcd 0x03 0xe8                (uint16, 3 bytes)
-5          → 0xfb                          (negative fixint, 1 byte)
-1000       → 0xd1 0xfc 0x18                (int16, 3 bytes)

Floating Point

Two formats are available:

Float32: 0xca followed by 4 bytes IEEE 754 single-precision, big-endian
Float64: 0xcb followed by 8 bytes IEEE 754 double-precision, big-endian

There is no fixfloat — even 0.0 requires 5 bytes (0xca 0x00 0x00 0x00 0x00) or 9 bytes for double precision. If you frequently transmit small floats, consider scaling them to integers on the application layer.

Nil, Boolean

nil:   0xc0
false: 0xc2
true:  0xc3

Booleans occupy exactly one byte each, unlike JSON's 4–5 bytes.

Strings

MessagePack treats strings as raw byte sequences. It is the application's responsibility to handle encoding (typically UTF-8). The format provides three string encodings:

Fixstr: 101xxxxx + x bytes (where xxxxx ranges 0–31, so 0xa0–0xbf)
Str8:   0xd9 + 1 byte length + data (up to 255 bytes)
Str16:  0xda + 2 bytes length + data (up to 65535 bytes)
Str32:  0xdb + 4 bytes length + data (up to 4294967295 bytes)

Example:

"hello" → 0xa5 0x68 0x65 0x6c 0x6c 0x6f
         ^--- 0xa5 = 0xa0 + 5 (fixstr of length 5)

Binary Data

Distinct from strings, binary blobs use a separate tag family to allow decoders to treat them differently (e.g., returning a byte array vs a string):

Bin8:  0xc4 + 1 byte length + data (up to 255 bytes)
Bin16: 0xc5 + 2 bytes length + data (up to 65535 bytes)
Bin32: 0xc6 + 4 bytes length + data (up to 4294967295 bytes)

Arrays

Fixarray: 1001xxxx + x elements (where xxxx ranges 0–15, so 0x90–0x9f)
Array16:  0xdc + 2 bytes element count + elements
Array32:  0xdd + 4 bytes element count + elements

An array of three fixints — [1, 2, 3] — encodes as:

0x93 0x01 0x02 0x03
 ^--- 0x93 = 0x90 + 3 (fixarray, 3 elements)

Maps

Fixmap: 1000xxxx + x*2 entries (where xxxx ranges 0–15, so 0x80–0x8f)
Map16:  0xde + 2 bytes entry count + key-value pairs
Map32:  0xdf + 4 bytes entry count + key-value pairs

A map is encoded as alternating keys and values. The pair count is half the number of elements on the wire. Example:

{"a": 1} → 0x81 0xa1 0x61 0x01
           ^fixmap(1) ^fixstr("a") ^fixint(1)

Extension Types

MessagePack reserves a family of tags for application-defined extensions:

Fixext1:  0xd4 + 1 byte type + 1 byte data
Fixext2:  0xd5 + 1 byte type + 2 bytes data
Fixext4:  0xd6 + 1 byte type + 4 bytes data
Fixext8:  0xd7 + 1 byte type + 8 bytes data
Fixext16: 0xd8 + 1 byte type + 16 bytes data
Ext8:     0xc7 + 1 byte length + 1 byte type + data
Ext16:    0xc8 + 2 bytes length + 1 byte type + data
Ext32:    0xc9 + 4 bytes length + 1 byte type + data

The type byte is an application-defined identifier (0–127 are reserved for MessagePack itself; 128–255 are available for user applications). This mechanism is how formats like MsgPack-RPC and Flutter's MessagePack codec encode timestamps and custom objects.

Timestamp Extension (Type -1 / 0xff)

The official specification defines an extension type for timestamps with nanosecond granularity:

Timestamp32: 0xd6 0xff + 4 bytes (32-bit Unix timestamp in seconds)
Timestamp64: 0xd7 0xff + 4 bytes nanoseconds + 4 bytes seconds (combined as 64-bit)
Timestamp96: 0xc7 0x0c 0xff + 4 bytes nanoseconds + 8 bytes seconds (96-bit full range)

This allows MessagePack to carry precise temporal data without string formatting overhead.

How to Use MessagePack

Using MessagePack is straightforward across virtually every programming language. Below are practical examples in the languages most commonly used in web and systems development.

Python

Install the official library. The package name on PyPI is msgpack (not msgpack-python — that is a legacy name):

pip install msgpack

Serialization and deserialization follow the same API shape as the built-in json module:

import msgpack

# Serialize a Python dict to MessagePack bytes
data = {
    "user_id": 42,
    "username": "alice",
    "tags": ["admin", "developer"],
    "active": True,
    "score": 97.5,
    "metadata": None
}

packed = msgpack.packb(data)
print(f"Packed size: {len(packed)} bytes")  # Significantly smaller than JSON

# Deserialize back to Python objects
unpacked = msgpack.unpackb(packed)
print(unpacked)
# Output: {'user_id': 42, 'username': 'alice', 'tags': ['admin', 'developer'], ...}

# Streaming unpacker for processing large streams
unpacker = msgpack.Unpacker()
with open("large_dataset.msgpack", "rb") as f:
    unpacker.feed(f.read())
    for record in unpacker:
        process(record)

The Python library automatically handles the distinction between strings and binary data. By default, decoded strings are returned as str (UTF-8 decoded). If you need raw bytes for binary fields, pass raw=True to unpackb or use the strict_map_key=False option for flexibility with non-string map keys.

JavaScript / Node.js

The official JavaScript implementation is @msgpack/msgpack:

npm install @msgpack/msgpack

The API is Promise-based and works in both Node.js and browsers:

import { encode, decode } from '@msgpack/msgpack';

const data = {
    userId: 42,
    username: "alice",
    tags: ["admin", "developer"],
    active: true,
    score: 97.5,
    metadata: null
};

// Encode to Uint8Array
const encoded = encode(data);
console.log(`Encoded size: ${encoded.length} bytes`);

// Decode back to JavaScript objects
const decoded = decode(encoded);
console.log(decoded);
// { userId: 42, username: 'alice', tags: ['admin', 'developer'], ... }

// Working with streams (Node.js)
import fs from 'fs';
const raw = fs.readFileSync('data.msgpack');
const object = decode(raw);

For browser usage, the library works natively with ArrayBuffer and Uint8Array, making it compatible with WebSocket binary frames and the Fetch API's arrayBuffer() method.

Go

Go has a popular third-party implementation called vmihailenco/msgpack (also known as github.com/vmihailenco/msgpack/v5):

go get github.com/vmihailenco/msgpack/v5

Usage with structs and maps:

package main

import (
    "fmt"
    "github.com/vmihailenco/msgpack/v5"
)

type User struct {
    UserID   int      `msgpack:"user_id"`
    Username string   `msgpack:"username"`
    Tags     []string `msgpack:"tags"`
    Active   bool     `msgpack:"active"`
    Score    float64  `msgpack:"score"`
}

func main() {
    user := User{
        UserID:   42,
        Username: "alice",
        Tags:     []string{"admin", "developer"},
        Active:   true,
        Score:    97.5,
    }

    // Marshal struct to MessagePack
    packed, err := msgpack.Marshal(user)
    if err != nil {
        panic(err)
    }
    fmt.Printf("Packed size: %d bytes\n", len(packed))

    // Unmarshal back
    var decoded User
    err = msgpack.Unmarshal(packed, &decoded)
    if err != nil {
        panic(err)
    }
    fmt.Printf("%+v\n", decoded)
}

Struct tags control field naming, and the library supports custom encoders/decoders via interfaces. For schema-less data, use interface{} and map[string]interface{} — the library will decode into Go's native types.

Rust

The Rust ecosystem uses rmp-serde (which integrates with Serde) for most MessagePack work:

[dependencies]
rmp-serde = "1.3"
serde = { version = "1.0", features = ["derive"] }

Usage mirrors standard Serde patterns:

use serde::{Serialize, Deserialize};
use rmp_serde::{encode, decode};

#[derive(Serialize, Deserialize, Debug)]
struct User {
    user_id: i32,
    username: String,
    tags: Vec<String>,
    active: bool,
    score: f64,
}

fn main() {
    let user = User {
        user_id: 42,
        username: "alice".to_string(),
        tags: vec!["admin".to_string(), "developer".to_string()],
        active: true,
        score: 97.5,
    };

    // Serialize
    let packed = encode::to_vec(&user).unwrap();
    println!("Packed size: {} bytes", packed.len());

    // Deserialize
    let decoded: User = decode::from_slice(&packed).unwrap();
    println!("{:?}", decoded);
}

For zero-copy deserialization of large payloads, consider rmp (the low-level library) which allows reading values directly from a buffer without intermediate allocations.

Java

The official Java implementation is msgpack-java:

// Maven dependency
// groupId: org.msgpack
// artifactId: msgpack-core
// version: 0.9.8
import org.msgpack.core.MessagePack;
import org.msgpack.core.MessagePacker;
import org.msgpack.core.MessageUnpacker;
import java.io.ByteArrayOutputStream;
import java.io.ByteArrayInputStream;

// Packing
ByteArrayOutputStream out = new ByteArrayOutputStream();
MessagePacker packer = MessagePack.newDefaultPacker(out);
packer.packMapHeader(3);
packer.packString("user_id").packInt(42);
packer.packString("username").packString("alice");
packer.packString("active").packBoolean(true);
packer.close();
byte[] packed = out.toByteArray();

// Unpacking
ByteArrayInputStream in = new ByteArrayInputStream(packed);
MessageUnpacker unpacker = MessagePack.newDefaultUnpacker(in);
int mapSize = unpacker.unpackMapHeader();
for (int i = 0; i < mapSize; i++) {
    String key = unpacker.unpackString();
    // Dispatch on key to unpack the appropriate value type
}
unpacker.close();

The Java library offers both a streaming API (shown above) and a higher-level msgpack-jackson module that integrates with Jackson for object mapping.

Best Practices

1. Choose Compact Representations for Your Data

MessagePack's fixint and fixstr encodings give you free compression for small values. Design your data schemas to keep integers small where possible. For example, use enumerated integer codes rather than string labels for status fields:

// Prefer this: 1 byte on the wire
status: 0x02  (fixint)

// Over this: 7+ bytes on the wire
status: "active"  (fixstr of length 6)

2. Prefer Arrays Over Maps for Homogeneous Collections

If you have a list of objects with identical keys, consider restructuring as a two-dimensional array (a column-oriented layout) rather than an array of maps. The savings from omitting repeated key strings can be substantial:

// Map-heavy: each row repeats keys
[{"id":1,"name":"Alice"},{"id":2,"name":"Bob"}]

// Column-oriented: keys appear once
{"columns":["id","name"],"rows":[[1,"Alice"],[2,"Bob"]]}

3. Use Binary Type for Raw Bytes, Not Strings

If your data contains cryptographic hashes, image thumbnails, or other non-text byte sequences, explicitly mark them as binary (bin8/bin16/bin32) rather than strings. This prevents accidental UTF-8 decoding errors and allows decoders to return appropriate byte-buffer types.

4. Stream When Data Is Unbounded

MessagePack's self-describing nature makes it ideal for streaming. You don't need to parse an entire document before accessing the first element. Use streaming unpackers (Unpacker in Python, MessageUnpacker in Java) when processing large files or WebSocket connections. This reduces memory pressure and improves latency.

5. Beware of Compatibility Pitfalls

Different MessagePack libraries have historically handled the string-vs-binary distinction differently. The specification was clarified in 2013 to separate strings from binary blobs, but older libraries may conflate them. Always test interoperability between your producer and consumer implementations. If you control both ends, use the latest library versions and explicitly test round-trip encoding.

6. Leverage Extension Types for Custom Objects

When you need to serialize domain objects that don't naturally map to MessagePack's built-in types (like datetime, UUID, or Decimal), use extension types rather than falling back to string representations. For example, a UUID can be encoded as a fixext16 with a custom type byte, saving significant space compared to a 36-character string.

7. Benchmark Your Specific Payloads

While MessagePack generally outperforms JSON, the magnitude of improvement depends heavily on your data's shape. Payloads dominated by long strings will see modest gains; payloads with many numbers and short keys will see dramatic improvements. Run benchmarks on representative production payloads to quantify the benefit in your context.

8. Consider Compatibility Mode for Web Browsers

If your application serves data directly to web frontends that expect JSON, you can either:

The first option gives you end-to-end MessagePack efficiency; the second preserves backward compatibility at the cost of gateway CPU.

Real-World Use Cases

MessagePack shines in several practical domains:

Tools and Ecosystem

Several tools support inspecting and debugging MessagePack data:

Conclusion

MessagePack occupies a unique and valuable niche in the serialization landscape. It combines the schema-less flexibility of JSON with the compactness and speed of binary formats, backed by a mature ecosystem spanning over 50 languages. The format's design — with its clever use of fixints, fixstrs, and fix-sized collections — means that common small values cost almost nothing on the wire, while the availability of 32-bit and 64-bit length prefixes ensures arbitrarily large data is handled gracefully.

For developers building APIs, microservices, real-time systems, or caching layers, MessagePack offers a pragmatic upgrade path: you can adopt it incrementally, converting one code path at a time, without the upfront investment of defining schemas or learning an IDL. The performance gains are measurable and often substantial, the learning curve is minimal, and the interoperability story is strong. Whether you're optimizing a WebSocket channel, shrinking your Redis cache footprint, or simply tired of parsing megabytes of repetitive JSON, MessagePack deserves a prominent spot in your toolkit.

🚀 Need a reliable AI agent for your project?

Deploy Hermes Agent in 10 minutes. Managed hosting, zero DevOps.

Get Started — $23.99/mo
← Back to all articles