Implementing CBOR Format: From Theory to Practice

What is CBOR?

CBOR (Concise Binary Object Representation) is a binary data serialization format defined in RFC 7049. It serves the same purpose as JSON — exchanging structured data between systems — but does so in a compact, binary form. Unlike JSON, which is text-based and must be parsed character by character, CBOR encodes values directly into bytes with a self-describing type system. This makes it highly efficient for constrained environments such as IoT devices, mobile sensors, and embedded systems where bandwidth and processing power are limited.

At its core, CBOR extends the simple data model of JSON (numbers, strings, arrays, maps, booleans, null) with additional native types: byte strings, definite and indefinite length collections, and tags that give semantic meaning to raw values. It also supports 64-bit integers, IEEE 754 floating-point numbers, and binary data without base64‑encoding overhead.

Why CBOR Matters

Choosing CBOR over JSON or other formats brings concrete benefits in real-world applications:

• Smaller payloads: Integer values and binary blobs are stored directly, avoiding the bloat of textual representation. This reduces network transfer time and storage costs.
• Faster parsing: A CBOR decoder reads the initial byte to determine the type and size, then copies or interprets subsequent bytes without tokenization or escape‑processing.
• No number ambiguity: CBOR distinguishes between integers, floating-point values, and big numbers. There’s no risk of losing precision when transporting 64‑bit integers or NaN/Infinity values.
• Native binary support: Byte strings are a first‑class type, eliminating the need to base64‑encode binary data, which typically adds 33% overhead.
• Streaming‑friendly: Indefinite‑length arrays and maps allow encoders to send data as it’s generated and decoders to process items incrementally.
• Self‑describing and extensible: Every CBOR item carries its own type information. Tags (IETF‑registered or application‑defined) let you annotate values with semantics like “epoch timestamp” or “regular expression”.

CBOR Data Model and Encoding Basics

The CBOR byte structure is simple and consistent. The first byte of any data item is divided into two parts: the top three bits encode the major type, and the lower five bits represent additional information. Major types are:

• 0 – unsigned integer
• 1 – negative integer (encoded as -1 - value)
• 2 – byte string
• 3 – text string (UTF-8)
• 4 – array of data items
• 5 – map of pairs (key/value data items)
• 6 – tag (semantic annotation)
• 7 – floating-point numbers and simple/break values

The additional information indicates the length of the payload (for strings, arrays, maps) or the integer value (when it fits in a small range). For example:

• Additional info 0–23 means the value is directly encoded in that field.
• 24 means an 8‑bit integer follows.
• 25 means a 16‑bit integer follows.
• 26 means a 32‑bit integer follows.
• 27 means a 64‑bit integer follows.

This encoding allows small integers to consume just one byte while still supporting up to full 64‑bit unsigned values.

Here’s how a few values are encoded:

0x00            → unsigned integer 0
0x01            → unsigned integer 1
0x18 0xff       → unsigned integer 255
0x19 0x03 0xe8  → unsigned integer 1000
0x20            → negative integer -1 (encoded as -1 - 0)
0x38 0x64       → negative integer -101 (encoded as -1 - 100, with 100 as 0x64)
0x41 0x41       → byte string "A" (length 1, then the byte 0x41)
0x61 0x41       → text string "A" (UTF-8)
0x80            → empty array (length 0)
0x9f            → start of indefinite-length array (ended with 0xff break)

How to Implement CBOR in Your Application

Choosing a Library

Most developers will integrate an existing CBOR library rather than writing a parser from scratch. Popular, well‑maintained implementations include:

• Python: cbor2 (pure‑Python and fast C extension), cbor
• JavaScript/Node.js: cbor npm package
• C/C++: tinycbor (small, portable), libcbor
• Go: fxamacker/cbor (fast, safe)
• Rust: serde_cbor

These libraries handle all low‑level encoding/decoding, provide APIs for tagged items, and often support streaming. The examples below use cbor2 for Python and cbor for JavaScript.

Encoding Data to CBOR (Python)

First install cbor2:

pip install cbor2

Then encode a Python dictionary containing mixed types:

import cbor2
import datetime

data = {
    "sensor_id": 42,
    "temperature": 23.5,
    "unit": "°C",
    "raw": b'\x01\x02\x03',          # byte string
    "timestamp": cbor2.CBORTag(1, datetime.datetime.now().timestamp())  # epoch tag
}

# Encode to bytes
cbor_bytes = cbor2.dumps(data)
print(cbor_bytes)  # b'\xa5...'

# Encode to hex for inspection
print(cbor_bytes.hex())

Key points:

• Python bytes becomes CBOR byte string (major type 2).
• Python int becomes CBOR integer (signed or unsigned as appropriate).
• Python float becomes CBOR float (usually 64‑bit IEEE).
• Dictionaries become CBOR maps.
• cbor2.CBORTag(1, value) wraps a value with tag number 1 (epoch time).

Encoding Data to CBOR (JavaScript / Node.js)

Install the cbor package:

npm install cbor

Encode a similar object:

const cbor = require('cbor');

const data = {
    sensor_id: 42,
    temperature: 23.5,
    unit: "°C",
    raw: Buffer.from([0x01, 0x02, 0x03]),
    timestamp: new cbor.Tagged(1, Math.floor(Date.now() / 1000))
};

// Encode synchronously to a Buffer
const encoded = cbor.encode(data);
console.log(encoded.toString('hex'));

Note that Buffer is used for byte strings, and cbor.Tagged applies the tag number 1.

Decoding CBOR Data

Decoding is symmetric to encoding. Here’s how to decode in Python:

import cbor2

# Assume cbor_bytes is from the previous example
decoded = cbor2.loads(cbor_bytes)

print(decoded["sensor_id"])   # 42
print(decoded["raw"])         # b'\x01\x02\x03'

# Access tagged value (the timestamp)
tag, epoch = decoded["timestamp"]
print(tag)    # 1
print(epoch)  # e.g. 1712345678.0 (float)

And in JavaScript:

const cbor = require('cbor');

// encoded is the Buffer from earlier
cbor.decodeFirst(encoded, (error, decoded) => {
    if (error) throw error;
    console.log(decoded.sensor_id);       // 42
    console.log(decoded.raw);             // Buffer <01 02 03>
    // The timestamp field will be a Tagged object
    const ts = decoded.timestamp;
    console.log(ts.tag);                  // 1
    console.log(ts.value);                // epoch as number
});

Working with Tags and Extensibility

Tags are a powerful CBOR feature. A tag wraps a data item with an integer identifier that tells decoders how to interpret the value. Standard tags are defined in the IANA CBOR Tag Registry. Some common ones:

• Tag 1: Epoch‑based date/time (number of seconds since 1970‑01‑01, may be integer or float).
• Tag 2: Bignum (unsigned, byte string in network byte order).
• Tag 3: Negative bignum.
• Tag 32: URI (text string).
• Tag 258: Set (array of unique items, mathematical set).

When encoding, you explicitly attach a tag. When decoding, the library returns a tagged object (e.g., CBORTag in Python, Tagged in JS). Your application logic can then branch on the tag value to interpret the payload correctly. This allows you to carry semantic meaning without inventing custom envelope structures.

Best Practices for CBOR Implementation

• Use deterministic encoding for security contexts: CBOR allows multiple encodings for the same data (e.g., different integer lengths). When signing or hashing, always apply canonical CBOR (RFC 7049 Section 3.9) to ensure a unique byte representation. Most libraries offer a canonical flag.
• Mind integer ranges: CBOR integers can go up to 2⁶⁴-1 (unsigned) or -2⁶⁴ (negative). If your language’s native integer type is smaller (e.g., JavaScript’s 53‑bit safe integer), use BigInt or libraries that handle 64‑bit values without loss.
• Validate untrusted input: A malicious CBOR payload could declare an enormous string or map length, causing memory exhaustion. Libraries should have configurable limits; always set them in production.
• Don’t blindly convert CBOR to JSON: CBOR maps can have keys of any type (byte strings, integers). JSON requires string keys. If you need JSON output, either restrict CBOR keys to text strings or write a custom serializer.
• Prefer definite-length encoding for simple data: Indefinite‑length arrays/maps are great for streaming but add complexity. Use them only when you genuinely need incremental processing.
• Handle tags explicitly: Decode tags into typed structures (e.g., DateTime objects) early in the pipeline, not ad‑hoc deep in business logic. This keeps interpretation consistent.
• Test interop: Different libraries may have slightly different default behaviors (e.g., float encoding width). Exchange test vectors with your partners to ensure compatibility.

CBOR vs. Other Binary Formats

CBOR sits alongside Protocol Buffers, MessagePack, and Avro. Each has strengths:

• CBOR is self‑describing (no schema needed), extensible via tags, and an IETF standard — ideal for open ecosystems and IoT.
• MessagePack is similar but lacks a formal tag registry and standardisation body.
• Protocol Buffers require a schema and code generation, yielding even smaller payloads and strong typing, but less flexible for ad‑hoc data.

Conclusion

Implementing CBOR in your application bridges the gap between the simplicity of JSON and the performance demands of modern distributed systems. With a solid library, encoding and decoding is as straightforward as working with JSON — just with bytes instead of strings. By understanding the major types, leveraging tags for semantic meaning, and following best practices around canonicalisation and input validation, you can build robust, interoperable components that thrive in constrained environments. Whether you’re building an IoT sensor network, a high‑throughput API, or a cross‑platform mobile app, CBOR is a practical, standardised choice that pays dividends in speed, size, and clarity.