← Back to DevBytes
AI July 20, 2026 13 min read

Embedding Models Compared: OpenAI vs Cohere vs Open Source

What Are Embeddings?

Embeddings are dense vector representations of text that capture semantic meaning in a form machines can process. Instead of treating words as discrete symbols, embeddings map them into a continuous vector space where similar concepts cluster together. A sentence, paragraph, or entire document gets compressed into a fixed-length array of floating-point numbers — typically ranging from 256 to 3072 dimensions.

Think of embeddings as a coordinate system for meaning. The phrase "king - man + woman" produces a vector near "queen" because the embedding space encodes relational semantics. This property makes embeddings the backbone of modern search, recommendation, clustering, and retrieval-augmented generation (RAG) systems.

How Embeddings Are Generated

Embedding models are trained on massive text corpora using objectives like contrastive learning or next-token prediction. During inference, the model processes input text through its encoder (transformer layers) and extracts the hidden state at a designated pooling layer. That hidden state becomes the embedding vector. No tokens are generated — just a numerical fingerprint of the input's semantic content.

Why Embeddings Matter for Developers

Embeddings bridge the gap between unstructured text and structured computation. Here's why they're indispensable:

Choosing the right embedding model affects latency, cost, accuracy, and the maximum dataset size you can index. The three main categories — proprietary cloud APIs (OpenAI, Cohere) and open source self-hosted models — each come with distinct tradeoffs.

OpenAI Embeddings

OpenAI offers two embedding models: text-embedding-3-small and text-embedding-3-large. Both support variable dimensionality via a dimensions parameter, letting you trade precision for storage savings. The small model produces embeddings up to 1536 dimensions, while the large model goes up to 3072 dimensions.

Key Characteristics

Code Example: Generating OpenAI Embeddings

import openai

openai.api_key = "your-api-key-here"

def get_openai_embeddings(texts, model="text-embedding-3-small", dimensions=512):
    """
    Generate embeddings for a list of texts using OpenAI.
    Returns a list of embedding vectors.
    """
    response = openai.Embedding.create(
        input=texts,
        model=model,
        dimensions=dimensions
    )
    # Extract embeddings in original order
    embeddings = [item.embedding for item in response.data]
    return embeddings

# Usage example
documents = [
    "The quick brown fox jumps over the lazy dog",
    "Machine learning transforms raw data into actionable insights",
    "Neural networks learn hierarchical representations"
]

embeddings = get_openai_embeddings(documents)
print(f"Generated {len(embeddings)} embeddings of dimension {len(embeddings[0])}")
# Output: Generated 3 embeddings of dimension 512

# Compute cosine similarity between first two documents
import numpy as np

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

sim = cosine_similarity(embeddings[0], embeddings[1])
print(f"Similarity: {sim:.4f}")

Using Reduced Dimensions

One of OpenAI's most practical features is the ability to request embeddings with fewer dimensions while preserving most of the quality. The model outputs a full vector internally, then applies a dimensionality reduction technique (Matryoshka representation) to return only the first N dimensions. This lets you shrink vectors by 75% (from 1536 to 384) with minimal accuracy loss.

# Request 256 dimensions instead of full 1536
embeddings_small = get_openai_embeddings(
    documents,
    model="text-embedding-3-small",
    dimensions=256
)
print(f"Vector size: {len(embeddings_small[0])}")  # 256

# Storage savings: 256 * 4 bytes (float32) = 1KB per vector
# vs 1536 * 4 bytes = 6KB per vector

Cohere Embeddings

Cohere provides the embed endpoint with multiple model variants optimized for different use cases. Their flagship embed-english-v3.0 and embed-multilingual-v3.0 models produce 1024-dimensional embeddings and support multiple input types: search documents, search queries, classification, and clustering. Each input type applies a different representation strategy tuned for the task.

Key Characteristics

Why Input Types Matter

Cohere's input type parameter is a distinguishing feature. When you embed a query with input_type="search_query" and documents with input_type="search_document", the model encodes them differently — queries receive a representation optimized for matching, while documents receive one optimized for being matched. This asymmetric encoding often yields better search relevance than symmetric approaches.

Code Example: Cohere Embeddings with Input Types

import cohere

co = cohere.Client("your-cohere-api-key")

# Embed documents (what you store in the vector DB)
documents = [
    "The Eiffel Tower is a wrought-iron lattice tower in Paris",
    "The Great Wall of China spans over 13,000 miles",
    "Mount Fuji is an active volcano and Japan's tallest peak"
]

doc_embeddings = co.embed(
    texts=documents,
    model="embed-english-v3.0",
    input_type="search_document"
).embeddings

# Embed a query (what the user types)
query = "Famous landmarks in France"
query_embedding = co.embed(
    texts=[query],
    model="embed-english-v3.0",
    input_type="search_query"
).embeddings[0]

# Compute similarity
import numpy as np

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

# Compare query against each document
for i, doc_emb in enumerate(doc_embeddings):
    sim = cosine_similarity(query_embedding, doc_emb)
    print(f"Document {i}: '{documents[i][:50]}...' — similarity: {sim:.4f}")

Classification and Clustering Modes

# Classification mode — embeddings optimized for training classifiers
texts = [
    "This product exceeded my expectations, highly recommend",
    "Terrible quality, broke after two days of use",
    "Shipping was fast but the color was slightly off"
]

classification_embeddings = co.embed(
    texts=texts,
    model="embed-english-v3.0",
    input_type="classification"
).embeddings

# These embeddings can now feed directly into sklearn classifiers
from sklearn.svm import SVC
from sklearn.preprocessing import LabelEncoder

labels = ["positive", "negative", "neutral"]
label_encoder = LabelEncoder()
y = label_encoder.fit_transform(labels)

# Train classifier on embeddings
clf = SVC(kernel="linear")
clf.fit(classification_embeddings, y)
print("Classifier trained on Cohere embeddings")

Open Source Embedding Models

Open source embedding models give you full control over deployment, latency, and data privacy. The ecosystem is rich, with models available via HuggingFace, the sentence-transformers library, and dedicated serving frameworks. Popular options include all-MiniLM-L6-v2 (384 dims, fast), BAAI/bge-large-en-v1.5 (1024 dims, high quality), and intfloat/e5-mistral-7b-instruct (massive, instruction-tuned).

Key Characteristics

Code Example: Local Embeddings with sentence-transformers

# Install: pip install sentence-transformers

from sentence_transformers import SentenceTransformer

# Load a lightweight model — 384 dimensions, extremely fast
model = SentenceTransformer("all-MiniLM-L6-v2")

documents = [
    "Embeddings turn text into numbers computers understand",
    "Vector search powers modern recommendation engines",
    "Open source models keep your data on your own servers",
    "The mitochondria is the powerhouse of the cell"
]

# Generate embeddings — this runs entirely locally
embeddings = model.encode(
    documents,
    normalize_embeddings=True,  # L2 normalize for cosine similarity
    show_progress_bar=True
)

print(f"Embedding shape: {embeddings.shape}")
# Output: Embedding shape: (4, 384)

# Semantic search with a query
query = "How do embeddings help search systems?"
query_embedding = model.encode([query], normalize_embeddings=True)[0]

# Compute similarities
from numpy import dot

scores = [dot(query_embedding, doc_emb) for doc_emb in embeddings]
# Since embeddings are normalized, dot product equals cosine similarity

for doc, score in zip(documents, scores):
    print(f"Score: {score:.3f} — {doc}")

Code Example: BGE Model with FlagEmbedding

# Install: pip install FlagEmbedding

from FlagEmbedding import FlagModel

# BGE-large — 1024 dimensions, state-of-the-art quality
model = FlagModel(
    "BAAI/bge-large-en-v1.5",
    query_instruction_for_retrieval="Represent this sentence for searching relevant passages: "
)

# Documents to index
documents = [
    "Python is a high-level programming language known for readability",
    "JavaScript powers interactive web applications in browsers",
    "Rust provides memory safety without garbage collection",
    "SQL is the standard language for relational database queries"
]

# Embed documents WITHOUT query instruction
doc_embeddings = model.encode(documents)

# Embed a query WITH query instruction (critical for BGE models)
query = "What language should I use for web development?"
query_embedding = model.encode_queries([query])[0]

# Compute similarity
import numpy as np

scores = (query_embedding @ doc_embeddings.T)
# Shape: (1, 4) after dot product

for i, score in enumerate(scores[0] if scores.ndim > 1 else scores):
    print(f"'{documents[i][:60]}' — similarity: {score:.3f}")

Code Example: Running Open Source Model as a Service

# Using FastAPI + sentence-transformers for a production embedding service
# Save as embedding_service.py and run: uvicorn embedding_service:app

from fastapi import FastAPI
from pydantic import BaseModel
from sentence_transformers import SentenceTransformer
import torch

app = FastAPI()

# Load model once at startup — use GPU if available
device = "cuda" if torch.cuda.is_available() else "cpu"
model = SentenceTransformer("all-MiniLM-L6-v2", device=device)

class EmbeddingRequest(BaseModel):
    texts: list[str]
    batch_size: int = 32

class EmbeddingResponse(BaseModel):
    embeddings: list[list[float]]
    dimensions: int
    count: int

@app.post("/embed", response_model=EmbeddingResponse)
def embed_texts(request: EmbeddingRequest):
    embeddings = model.encode(
        request.texts,
        batch_size=request.batch_size,
        normalize_embeddings=True,
        show_progress_bar=False
    )
    return EmbeddingResponse(
        embeddings=embeddings.tolist(),
        dimensions=embeddings.shape[1],
        count=len(request.texts)
    )

@app.get("/health")
def health():
    return {"status": "ok", "device": device}

# Client usage:
# import requests
# resp = requests.post("http://localhost:8000/embed", json={"texts": ["hello world"]})
# print(resp.json()["embeddings"])

Head-to-Head Comparison

Latency and Throughput

Latency breaks down into network overhead plus model inference time. OpenAI and Cohere add roughly 50–200ms of network round-trip latency depending on your region. Open source models running locally can achieve 1–10ms inference on GPU, but require upfront infrastructure investment. For real-time applications with strict p99 latency budgets (under 30ms), self-hosting is often the only viable path.

Cost Analysis at Scale

Let's run the numbers for 1 million embeddings:

OpenAI wins on simplicity for low-to-medium volume. Open source wins on marginal cost at high volume. Cohere sits in the middle but offers unique value through input-type optimization and multilingual support.

Quality Benchmarks (MTEB Leaderboard)

The Massive Text Embedding Benchmark (MTEB) provides standardized evaluation across retrieval, clustering, classification, and more. As of mid-2024:

The quality gap between the best open source models and proprietary APIs has narrowed dramatically. For many applications, the difference is negligible compared to the benefits of data locality and cost control.

Best Practices for Embedding Workflows

1. Normalize Embeddings

Always L2-normalize embeddings before storing them. This converts Euclidean distance to cosine similarity (or dot product) and simplifies nearest-neighbor search. Most libraries do this by default, but verify:

# Explicit normalization
from sklearn.preprocessing import normalize

embeddings = normalize(embeddings, norm='l2')
# Now dot(emb_a, emb_b) == cosine_similarity(emb_a, emb_b)

2. Batch Strategically

All embedding APIs and models benefit from batching. For APIs, batch at the maximum allowed size (2048 for OpenAI, 96 for Cohere) to minimize round-trips. For local models, batch to GPU memory capacity — typically 64–256 texts per batch depending on text length and model size.

# Smart batching for local inference
def embed_in_batches(model, texts, batch_size=64):
    all_embeddings = []
    for i in range(0, len(texts), batch_size):
        batch = texts[i:i + batch_size]
        batch_embeddings = model.encode(batch, normalize_embeddings=True)
        all_embeddings.append(batch_embeddings)
    return np.vstack(all_embeddings)

3. Choose Dimensions Wisely

Higher dimensions capture more nuance but consume more memory and slow down similarity search. Start with the smallest dimensions that meet your accuracy requirements. OpenAI's Matryoshka embeddings let you test multiple dimensionalities from a single API call (embed at 1536, then truncate to 256, 512, etc.).

# Experiment with different dimensions from a single embedding
embeddings_full = get_openai_embeddings(documents, dimensions=1536)

# Truncate and evaluate
for dims in [256, 512, 768, 1024]:
    truncated = [emb[:dims] for emb in embeddings_full]
    # Run your evaluation metric here
    print(f"Evaluating at {dims} dimensions...")

4. Use Appropriate Preprocessing

For search applications, prepend instructional prefixes when the model expects them. BGE models require "Represent this sentence for searching relevant passages:" on queries. E5 models need "query:" and "passage:" prefixes. Always consult the model card.

# Model-specific preprocessing
def preprocess_for_model(texts, model_name, is_query=False):
    if "bge" in model_name.lower():
        if is_query:
            prefix = "Represent this sentence for searching relevant passages: "
            return [prefix + t for t in texts]
    elif "e5" in model_name.lower():
        prefix = "query: " if is_query else "passage: "
        return [prefix + t for t in texts]
    return texts  # Most models need no prefix

# Example: BGE query preprocessing
query_raw = "How does photosynthesis work?"
query_processed = preprocess_for_model([query_raw], "bge-large-en-v1.5", is_query=True)
print(query_processed[0])
# "Represent this sentence for searching relevant passages: How does photosynthesis work?"

5. Cache Embeddings Aggressively

Embedding computation is expensive and deterministic — the same input always produces the same vector. Cache embeddings at every layer: use an in-memory LRU cache for hot queries, persist embeddings alongside source documents, and version your embedding model so you can rebuild caches when models change.

import hashlib
import json
from functools import lru_cache

# Simple file-based embedding cache
class EmbeddingCache:
    def __init__(self, cache_path="embedding_cache.json"):
        self.cache_path = cache_path
        self.cache = self._load()

    def _load(self):
        try:
            with open(self.cache_path) as f:
                return json.load(f)
        except FileNotFoundError:
            return {}

    def _save(self):
        with open(self.cache_path, "w") as f:
            json.dump(self.cache, f)

    def get_or_compute(self, text, model_name, compute_fn):
        key = hashlib.md5(f"{model_name}:{text}".encode()).hexdigest()
        if key in self.cache:
            return self.cache[key]
        embedding = compute_fn(text)
        self.cache[key] = embedding
        if len(self.cache) % 100 == 0:
            self._save()
        return embedding

# Usage
cache = EmbeddingCache()
embedding = cache.get_or_compute(
    "neural networks explained",
    "text-embedding-3-small",
    lambda t: get_openai_embeddings([t])[0]
)

6. Monitor Embedding Drift

When you update embedding models, vectors change. Maintain a model version tag alongside every stored embedding. Before switching versions, sample 1000 queries and measure recall overlap between old and new embeddings. If recall drops below your threshold, rebuild the index incrementally.

# Versioning embeddings in a vector database context
import datetime

class VersionedEmbeddingStore:
    def __init__(self):
        self.model_version = "text-embedding-3-small-v1"
        self.store = {}  # doc_id -> {embedding, model_version, created_at}

    def add(self, doc_id, embedding):
        self.store[doc_id] = {
            "embedding": embedding,
            "model_version": self.model_version,
            "created_at": datetime.datetime.now().isoformat()
        }

    def check_version_consistency(self):
        versions = set(v["model_version"] for v in self.store.values())
        if len(versions) > 1:
            print(f"WARNING: Multiple embedding versions detected: {versions}")
        return len(versions) == 1

Choosing the Right Approach: Decision Framework

Your choice depends on four factors. Here's a practical decision matrix:

Conclusion

Embedding models have become a commodity in the best sense — multiple high-quality options compete on price, speed, and specialization rather than fundamental capability gaps. OpenAI delivers the smoothest developer experience with its Matryoshka dimensionality trick and generous batch limits. Cohere carves a niche with asymmetric encoding and task-aware input types that genuinely improve retrieval and classification workflows. Open source models, led by BGE and the sentence-transformers ecosystem, now match proprietary quality while offering zero data egress, sub-millisecond latency, and the freedom to fine-tune. The right choice depends on your scale, latency budget, and privacy requirements — but the good news is that migrating between them requires only regenerating embeddings, not rearchitecting your entire pipeline. Start with the simplest integration that meets your needs, cache aggressively, version your embeddings, and you'll build a robust semantic layer that scales gracefully.

— Ad —

Google AdSense will appear here after approval

← Back to all articles