← Back to DevBytes
Developer Tutorial July 11, 2026 15 min read

Implementing Ion Format: From Theory to Practice

Introduction to Amazon Ion

Amazon Ion is a richly-typed, self-describing data serialization format developed by Amazon and open-sourced under the Apache 2.0 license. At its core, Ion is a superset of JSON — any valid JSON document is also a valid Ion document — but it extends JSON's limited type system with a much broader set of native data types, annotations, and an efficient binary encoding. Ion was designed to address the shortcomings of JSON in large-scale, high-performance systems while maintaining human-readability when needed.

Ion comes in two interchangeable representations: a text format that is easy to read and author by humans, and a binary format that is compact, efficient to parse, and ideal for machine-to-machine communication. Both representations share the same data model, meaning you can convert losslessly between text and binary Ion without any ambiguity.

Why Ion Matters

🚀 Deploy your AI agent in 10 minutes

Managed Hermes hosting. Zero DevOps. 100M tokens/mo included.

Try it free →

Before diving into implementation, it's worth understanding the concrete problems Ion solves and why you might choose it over alternatives like JSON, MessagePack, or Protocol Buffers.

1. A Richer Type System

JSON's type system is limited to strings, numbers (with no distinction between integers and floats), booleans, arrays, objects, and null. Ion expands this dramatically:

2. Annotations: Metadata Without Pollution

Ion allows any value to carry one or more annotations — metadata tags that qualify the value's meaning without altering the underlying data. For example, a struct field can be annotated with $id to mark it as a unique identifier, or a timestamp can carry an precision::seconds annotation. Annotations are a first-class concept in Ion, not a hack layered on top of string conventions.

3. Efficient Binary Encoding

The Ion binary format uses a compact, self-describing wire encoding. Unlike JSON, which must be parsed character-by-character with backtracking, binary Ion can be read with simple table-driven state machines. The binary format encodes type information directly into the bytes, enabling readers to skip entire values without fully parsing them — a technique called skip-scanning that is invaluable for selective data access in large streams.

4. Self-Describing Nature

Every Ion value carries its own type code. There is no need for an external schema to interpret the data. This makes Ion ideal for systems where data evolves independently of consuming applications, such as event logs, message buses, and document databases.

Core Concepts of the Ion Data Model

Before writing code, let's establish the foundational data model. Ion values form a hierarchy with a single root value. That root can be any Ion type — a struct, a list, a scalar, or even a null.

Scalar Types

Ion defines the following scalar (atomic) types:

Container Types

Annotations in Practice

Annotations appear before a value, separated by ::. Multiple annotations are separated by spaces:

{
  person::{
    firstName::"Jane",
    lastName::"Doe",
    $id::"urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6",
    precision::seconds::modified::2024-03-15T10:30:00Z
  }
}

Here, person annotates the struct as a person entity, $id marks a field as a unique identifier, and precision::seconds combined with modified qualifies the timestamp's precision and its role as a modification time.

Getting Started: Setting Up Ion in Your Project

The reference Ion implementation is written in Java and is available via Maven Central. Amazon also maintains official libraries for Rust, Python, C, and JavaScript/TypeScript. We'll use Java for the core examples and show Python snippets for contrast.

Java (Maven)

<dependency>
  <groupId>com.amazon.ion</groupId>
  <artifactId>ion-java</artifactId>
  <version>1.11.0</version>
</dependency>

Python

pip install amazon-ion

Rust (Cargo.toml)

[dependencies]
ion-rs = "0.18"

Reading and Writing Ion Text

The text format is the ideal starting point because it is directly readable and mirrors the concepts of the data model most transparently. Let's begin with basic read/write operations in Java.

Writing Ion Text in Java

We use an IonWriter to produce Ion data. The writer is a streaming API — you call methods to emit values in order, nesting containers as you go.

import com.amazon.ion.IonWriter;
import com.amazon.ion.IonSystem;
import com.amazon.ion.system.IonSystemBuilder;
import java.io.ByteArrayOutputStream;

public class IonTextWriterExample {
    public static void main(String[] args) throws Exception {
        // Obtain an IonSystem instance — the entry point for all Ion operations
        IonSystem system = IonSystemBuilder.standard().build();

        ByteArrayOutputStream out = new ByteArrayOutputStream();
        
        // Create a text writer (human-readable output)
        try (IonWriter writer = system.newTextWriter(out)) {
            // Start a struct (top-level object)
            writer.stepIn(IonType.STRUCT);
            
            // Write a string field
            writer.setFieldName("name");
            writer.writeString("Ada Lovelace");
            
            // Write an integer field
            writer.setFieldName("birthYear");
            writer.writeInt(1815);
            
            // Write a decimal field
            writer.setFieldName("contributionScore");
            writer.writeDecimal(BigDecimal.valueOf(97.5));
            
            // Write a timestamp field
            writer.setFieldName("recordedAt");
            writer.writeTimestamp("2024-03-15T10:30:00.000Z");
            
            // Write an annotated field
            writer.setFieldName("id");
            writer.writeSymbol("$id");  // annotation
            writer.writeString("urn:uuid:a1b2c3d4-e5f6-7890-abcd-ef1234567890");
            
            // Step out of the struct
            writer.stepOut();
        }
        
        String ionText = out.toString();
        System.out.println(ionText);
    }
}

This produces:

{name:"Ada Lovelace",birthYear:1815,contributionScore:97.5,recordedAt:2024-03-15T10:30:00.000Z,$id::"urn:uuid:a1b2c3d4-e5f6-7890-abcd-ef1234567890"}

Reading Ion Text in Java

The IonReader is a cursor-based streaming API. It advances through values, reporting the current type and allowing you to extract the data.

import com.amazon.ion.IonReader;
import com.amazon.ion.IonSystem;
import com.amazon.ion.system.IonSystemBuilder;

public class IonTextReaderExample {
    public static void main(String[] args) {
        IonSystem system = IonSystemBuilder.standard().build();
        
        String ionText = "{name:\"Ada Lovelace\",birthYear:1815," +
                         "contributionScore:97.5," +
                         "recordedAt:2024-03-15T10:30:00.000Z}";
        
        try (IonReader reader = system.newReader(ionText)) {
            // The reader starts positioned before the first value
            // IonType indicates the type of the current value
            reader.next();  // advances into the struct
            System.out.println("Top-level type: " + reader.getType());  // STRUCT
            
            // Step into the struct to iterate its fields
            reader.stepIn();
            
            while (reader.next() != null) {
                String fieldName = reader.getFieldName();
                IonType type = reader.getType();
                
                System.out.println("Field: " + fieldName + " (type: " + type + ")");
                
                // Extract value based on type
                switch (type) {
                    case STRING:
                        System.out.println("  Value: " + reader.stringValue());
                        break;
                    case INT:
                        System.out.println("  Value: " + reader.intValue());
                        break;
                    case DECIMAL:
                        System.out.println("  Value: " + reader.decimalValue());
                        break;
                    case TIMESTAMP:
                        System.out.println("  Value: " + reader.timestampValue());
                        break;
                    default:
                        System.out.println("  (skipping type: " + type + ")");
                }
            }
            
            reader.stepOut();  // exit the struct
        }
    }
}

Python Example: Writing and Reading Ion

The Python library provides a similar streaming API with a slightly different flavor:

from amazon.ion import simpleion
import datetime

# Writing Ion text
ion_data = {
    "name": "Ada Lovelace",
    "birthYear": 1815,
    "contributionScore": 97.5,
    "recordedAt": datetime.datetime(2024, 3, 15, 10, 30, 0, tzinfo=datetime.timezone.utc)
}

# Serialize to Ion text
ion_text = simpleion.dumps(ion_data, binary=False)
print(ion_text)

# Reading Ion text
loaded = simpleion.loads(ion_text)
print(loaded["name"])        # 'Ada Lovelace'
print(loaded["birthYear"])   # 1815

Working with Ion Binary

The binary format is what makes Ion shine in production. It uses a compact type-length-value encoding that is self-describing yet highly efficient. The key insight is that every byte in the binary stream carries enough type information for a reader to know exactly how to interpret the subsequent bytes — no schema required.

Binary Encoding Overview

Each Ion binary value begins with an opcode byte that encodes the type and, for small values, the length inline. For example:

Writing Binary Ion in Java

The only change from the text writer example is using newBinaryWriter instead of newTextWriter:

ByteArrayOutputStream binaryOut = new ByteArrayOutputStream();

try (IonWriter writer = system.newBinaryWriter(binaryOut)) {
    writer.stepIn(IonType.STRUCT);
    
    writer.setFieldName("name");
    writer.writeString("Ada Lovelace");
    
    writer.setFieldName("birthYear");
    writer.writeInt(1815);
    
    writer.setFieldName("active");
    writer.writeBool(true);
    
    writer.stepOut();
}

byte[] binaryIon = binaryOut.toByteArray();
System.out.println("Binary Ion length: " + binaryIon.length + " bytes");

// The binary can be read back identically
try (IonReader reader = system.newReader(binaryIon)) {
    reader.next();
    reader.stepIn();
    
    while (reader.next() != null) {
        System.out.println("Field: " + reader.getFieldName() + 
                           " = " + reader.asString());
    }
    reader.stepOut();
}

Comparing Text and Binary Size

For the same data, binary Ion is dramatically smaller. Here's a practical comparison:

// Same data in both formats
String ionText = "{id:1,firstName:\"John\",lastName:\"Smith\"," +
                  "email:\"john@example.com\",age:35,balance:1234.56," +
                  "active:true,tags:[\"premium\",\"verified\"]}";

// Text size
byte[] textBytes = ionText.getBytes(StandardCharsets.UTF_8);
System.out.println("Text Ion: " + textBytes.length + " bytes");

// Binary size
ByteArrayOutputStream bos = new ByteArrayOutputStream();
try (IonWriter binWriter = system.newBinaryWriter(bos)) {
    binWriter.stepIn(IonType.STRUCT);
    binWriter.setFieldName("id"); binWriter.writeInt(1);
    binWriter.setFieldName("firstName"); binWriter.writeString("John");
    binWriter.setFieldName("lastName"); binWriter.writeString("Smith");
    binWriter.setFieldName("email"); binWriter.writeString("john@example.com");
    binWriter.setFieldName("age"); binWriter.writeInt(35);
    binWriter.setFieldName("balance"); 
    binWriter.writeDecimal(new java.math.BigDecimal("1234.56"));
    binWriter.setFieldName("active"); binWriter.writeBool(true);
    binWriter.setFieldName("tags");
    binWriter.stepIn(IonType.LIST);
    binWriter.writeString("premium");
    binWriter.writeString("verified");
    binWriter.stepOut();
    binWriter.stepOut();
}
System.out.println("Binary Ion: " + bos.size() + " bytes");
// Typical output: Text Ion: ~120 bytes, Binary Ion: ~70 bytes

Advanced Features

Symbol Tables: Efficient Identifier Encoding

Symbols are one of Ion's most powerful features. Unlike strings, symbols are interned — identical symbols share the same identity. In the binary format, symbols are encoded as small integers that index into a symbol table. This means repeated field names and enumerated values become single-byte references.

Ion uses a system symbol table (pre-populated with common strings like "id", "name", "type") and allows custom symbol tables to be prepended to data streams.

// Demonstrating symbol internment
IonSystem system = IonSystemBuilder.standard().build();

ByteArrayOutputStream out = new ByteArrayOutputStream();
try (IonWriter writer = system.newBinaryWriter(out)) {
    // The writer automatically interns symbols in the local symbol table
    writer.stepIn(IonType.STRUCT);
    
    // These field names become symbol references
    writer.setFieldName("userId");     // becomes symbol index
    writer.writeString("user-001");
    writer.setFieldName("userId");     // same symbol, same index
    writer.writeString("user-002");
    
    writer.stepOut();
}

// When reading, symbols are resolved transparently
try (IonReader reader = system.newReader(out.toByteArray())) {
    reader.next();
    reader.stepIn();
    while (reader.next() != null) {
        String fieldName = reader.getFieldName();  // resolved symbol
        System.out.println(fieldName);
    }
}

Annotations on Values

Annotations in the reader/writer API are handled with dedicated methods. Annotations must be written before the value they annotate:

// Writing annotated values
try (IonWriter writer = system.newTextWriter(System.out)) {
    writer.stepIn(IonType.STRUCT);
    
    // Add annotations before writing the value
    writer.setFieldName("eventTime");
    writer.addTypeAnnotation("precision");
    writer.addTypeAnnotation("seconds");
    writer.writeTimestamp("2024-03-15T10:30:00Z");
    
    // Annotating a struct field as a unique identifier
    writer.setFieldName("id");
    writer.addTypeAnnotation("$id");
    writer.writeString("urn:uuid:a1b2c3d4-e5f6-7890");
    
    writer.stepOut();
}

// Reading annotations
String ionText = "{eventTime:precision::seconds::2024-03-15T10:30:00Z," +
                  "$id::\"urn:uuid:a1b2c3d4-e5f6-7890\"}";
try (IonReader reader = system.newReader(ionText)) {
    reader.next();
    reader.stepIn();
    while (reader.next() != null) {
        // Get annotations as a string array
        String[] annotations = reader.getTypeAnnotations();
        if (annotations.length > 0) {
            System.out.println("Annotations on " + reader.getFieldName() + ":");
            for (String ann : annotations) {
                System.out.println("  - " + ann);
            }
        }
    }
}

Working with Timestamps and Decimals

Ion's temporal and numeric precision is best-in-class. Here's how to leverage it correctly:

// Precise timestamp handling
import com.amazon.ion.Timestamp;

// Nanosecond precision timestamp
Timestamp precise = Timestamp.valueOf("2024-03-15T10:30:00.123456789Z", null);
// Access components
System.out.println("Year: " + precise.getYear());
System.out.println("Month: " + precise.getMonth());
System.out.println("Nanoseconds: " + precise.getSecondFractionAsInt());

// Writing with full precision
try (IonWriter writer = system.newTextWriter(System.out)) {
    writer.writeTimestamp(precise);
}

// Arbitrary-precision decimals
import java.math.BigDecimal;

// Decimals with high precision — no floating-point rounding
BigDecimal price = new BigDecimal("1234.56789012345678901234567890");
try (IonWriter writer = system.newTextWriter(System.out)) {
    writer.writeDecimal(price);
}
// Output: 1234.56789012345678901234567890 — no precision loss

Best Practices

1. Choose the Right Format for the Context

2. Leverage Symbols for Repeated Strings

If your data contains frequently repeated field names, status values, or enumerated codes, symbols reduce storage size and comparison cost. The binary format automatically interns symbols, so the cost is near-zero.

// Good: symbol for status
writer.setFieldName("status");
writer.writeSymbol("active");  // interned, efficient

// Less efficient: string for status
writer.setFieldName("status");
writer.writeString("active");  // stored as full UTF-8 bytes

3. Use Annotations Instead of Type-Discriminator Fields

Instead of adding a "type": "person" field to distinguish entity types, use an annotation on the struct itself. This keeps the data model clean and the discriminator is parseable without entering the struct.

// Preferred: annotation-based discrimination
person::{name:"Jane", age:30}
organization::{name:"Acme", employees:500}

// Avoid: type-as-field pattern
{type:"person", name:"Jane", age:30}

4. Prefer Decimal for Financial Data

Never use float for monetary values. Ion's decimal type uses arbitrary-precision fixed-point arithmetic, avoiding the rounding errors inherent in IEEE-754 floating point.

// Correct: decimal for money
writer.writeDecimal(new BigDecimal("100.50"));

// Incorrect: float for money — will drift over computations
writer.writeFloat(100.50);

5. Handle Nulls Explicitly by Type

Ion's typed nulls let you distinguish between "the value is absent" and "the value is absent but we know its intended type." When reading, always check for null before extracting:

while (reader.next() != null) {
    if (reader.isNull()) {
        System.out.println("Field " + reader.getFieldName() + 
                           " is null of type " + reader.getType());
        continue;
    }
    // Safe to extract now
    String value = reader.stringValue();
}

6. Stream Large Datasets

Both the reader and writer are streaming APIs. For large datasets (gigabytes of Ion data), never load the entire document into memory. Process values as they arrive:

// Streaming through a large Ion file
InputStream largeFile = new FileInputStream("massive-dataset.ion");
try (IonReader reader = system.newReader(largeFile)) {
    // Process top-level values one at a time
    while (reader.next() != null) {
        // Each top-level value is processed and discarded
        processSingleRecord(reader);
    }
    // Memory usage stays constant regardless of file size
}

7. Use the DOM API for Simple Cases, Streaming for Performance

Ion Java offers two API styles: the streaming IonReader/IonWriter (shown above) and a DOM-style IonValue tree API. The DOM API is convenient for small documents:

// DOM-style loading — simple but loads entire document into memory
IonValue tree = system.newLoader().load("{name:\"Alice\",age:30}");
IonStruct struct = (IonStruct) tree;
String name = ((IonString) struct.get("name")).stringValue();
System.out.println("Name: " + name);

For production workloads with large or numerous documents, prefer the streaming API to keep memory bounded.

Putting It All Together: A Complete Example

Here is a complete, self-contained example that demonstrates writing binary Ion to a file, reading it back with streaming, handling annotations, and properly managing resources:

import com.amazon.ion.*;
import com.amazon.ion.system.IonSystemBuilder;
import java.io.*;
import java.math.BigDecimal;
import java.nio.file.*;

public class CompleteIonExample {
    
    public static void main(String[] args) throws Exception {
        IonSystem system = IonSystemBuilder.standard().build();
        Path filePath = Paths.get("example.ion");
        
        // ---- WRITE PHASE ----
        // Write binary Ion with annotations, symbols, and precise decimals
        try (OutputStream fos = Files.newOutputStream(filePath);
             IonWriter writer = system.newBinaryWriter(fos)) {
            
            // Write as a top-level struct (document)
            writer.stepIn(IonType.STRUCT);
            
            // Metadata fields with annotations
            writer.setFieldName("documentId");
            writer.addTypeAnnotation("$id");
            writer.writeString("doc-2024-001");
            
            writer.setFieldName("createdAt");
            writer.addTypeAnnotation("precision");
            writer.addTypeAnnotation("milliseconds");
            writer.writeTimestamp("2024-03-15T10:30:00.123Z");
            
            // Nested struct for address
            writer.setFieldName("billingAddress");
            writer.stepIn(IonType.STRUCT);
            writer.setFieldName("street");
            writer.writeString("123 Main St");
            writer.setFieldName("city");
            writer.writeString("Seattle");
            writer.setFieldName("zipCode");
            writer.writeString("98101");
            writer.stepOut();  // exit billingAddress
            
            // A list of line items with precise decimals
            writer.setFieldName("lineItems");
            writer.stepIn(IonType.LIST);
            
            // Item 1
            writer.stepIn(IonType.STRUCT);
            writer.setFieldName("description");
            writer.writeString("Widget Pro");
            writer.setFieldName("unitPrice");
            writer.writeDecimal(new BigDecimal("29.99"));
            writer.setFieldName("quantity");
            writer.writeInt(3);
            writer.stepOut();
            
            // Item 2
            writer.stepIn(IonType.STRUCT);
            writer.setFieldName("description");
            writer.writeString("Extended Warranty");
            writer.setFieldName("unitPrice");
            writer.writeDecimal(new BigDecimal("149.99"));
            writer.setFieldName("quantity");
            writer.writeInt(1);
            writer.stepOut();
            
            writer.stepOut();  // exit lineItems
            
            // Use symbol for status (interned, efficient)
            writer.setFieldName("status");
            writer.writeSymbol("processed");
            
            writer.stepOut();  // exit top-level struct
        }
        
        System.out.println("Binary Ion written to: " + filePath);
        System.out.println("File size: " + Files.size(filePath) + " bytes");
        
        // ---- READ PHASE ----
        // Stream-read the binary Ion file
        try (InputStream fis = Files.newInputStream(filePath);
             IonReader reader = system.newReader(fis)) {
            
            // Advance to the first (and only) top-level value
            IonType topType = reader.next();
            if (topType == null) {
                System.out.println("Empty file!");
                return;
            }
            System.out.println("\nTop-level type: " + topType);
            
            // Step into the struct
            reader.stepIn();
            
            while (reader.next() != null) {
                String fieldName = reader.getFieldName();
                String[] annotations = reader.getTypeAnnotations();
                IonType type = reader.getType();
                
                // Print annotations if present
                String annStr = annotations.length > 0 
                    ? " (" + String.join("::", annotations) + ") " 
                    : "";
                
                System.out.println("Field: " + fieldName + annStr + 
                                   "[type: " + type + "]");
                
                if (reader.isNull()) {
                    System.out.println("  Value: null." + type);
                    continue;
                }
                
                // Dispatch on type
                switch (type) {
                    case STRING:
                        System.out.println("  Value: " + reader.stringValue());
                        break;
                    case SYMBOL:
                        System.out.println("  Value (symbol): " + 
                                           reader.symbolValue());
                        break;
                    case INT:
                        System.out.println("  Value: " + reader.intValue());
                        break;
                    case DECIMAL:
                        System.out.println("  Value: " + 
                                           reader.decimalValue().toPlainString());
                        break;
                    case TIMESTAMP:
                        System.out.println("  Value: " + 
                                           reader.timestampValue().toString());
                        break;
                    case STRUCT:
                        System.out.println("  (nested struct — stepping in)");
                        reader.stepIn();
                        while (reader.next() != null) {
                            System.out.println("    " + reader.getFieldName() + 
                                               ": " + reader.asString());
                        }
                        reader.stepOut();
                        break;
                    case LIST:
                        System.out.println("  (list — stepping in)");
                        reader.stepIn();
                        int idx = 0;
                        while (reader.next() != null) {
                            System.out.println("    [" + idx + "] type: " + 
                                               reader.getType());
                            if (reader.getType() == IonType.STRUCT) {
                                reader.stepIn();
                                while (reader.next() != null) {
                                    System.out.println("      " + 
                                        reader.getFieldName() + ": " + 
                                        reader.asString());
                                }
                                reader.stepOut();
                            }
                            idx++;
                        }
                        reader.stepOut();
                        break;
                    default:
                        System.out.println("  Value: " + reader.asString());
                }
            }
            
            reader.stepOut();  // exit top-level struct
        }
        
        System.out.println("\n--- Reading complete ---");
    }
}

Running this program produces structured, navigable output that demonstrates every concept discussed: annotations, symbols, nested containers, typed nulls, and precise numeric handling — all from a compact binary file.

Conclusion

Amazon Ion bridges the gap between human-friendly data formats like JSON and high-performance binary protocols like Protocol Buffers, offering the best of both worlds in a single, unified data model. Its rich type system eliminates the precision and expressiveness limitations of JSON, while its binary encoding provides the compactness and parse-efficiency demanded by modern distributed systems. By mastering the streaming reader/writer API, understanding annotations and symbols, and following the best practices outlined here, you can build data pipelines that are simultaneously more expressive, more efficient, and more maintainable. Whether you're building event-driven architectures, document databases, or high-throughput message systems, Ion provides a robust, future-proof foundation for your data serialization needs.

🚀 Need a reliable AI agent for your project?

Deploy Hermes Agent in 10 minutes. Managed hosting, zero DevOps.

Get Started — $23.99/mo
← Back to all articles