Introduction to Hybrid Search in Elasticsearch
Retrieval-Augmented Generation (RAG) has become the standard architecture for building LLM-powered applications that ground their answers in private or domain-specific data. At the heart of every RAG system is a retrieval step, and the quality of that retrieval directly determines the quality of the final answer. Hybrid search—the combination of lexical (keyword) and semantic (vector) search—is one of the most effective ways to maximize retrieval quality in Elasticsearch.
In this tutorial, you'll learn what hybrid search is, why it matters for RAG, and how to implement it end-to-end in Elasticsearch using Reciprocal Rank Fusion (RRF). We'll cover index mapping, document ingestion, query construction, and integration with an LLM. Every code example is complete and ready to run.
What Is Hybrid Search?
Hybrid search combines two complementary retrieval strategies:
- Lexical search (BM25): Matches documents based on exact term overlap, token frequency, and inverse document frequency. It excels at matching specific keywords, names, IDs, codes, and rare terms.
- Semantic search (kNN / dense vectors): Matches documents based on the meaning of the text, computed via embeddings. It excels at matching paraphrases, synonyms, and conceptual similarity even when no words overlap.
Each approach has blind spots. BM25 fails when the user uses different vocabulary than the document. Vector search can miss exact matches and may rank a semantically similar but factually wrong document highly. Hybrid search runs both retrievers and merges their results, getting the best of both worlds.
Reciprocal Rank Fusion (RRF)
Elasticsearch merges the two result sets using Reciprocal Rank Fusion, a rank-based fusion algorithm. RRF doesn't require score calibration between the two retrievers—it only uses the rank position of each document in each result set. The formula is:
rrf_score(d) = sum over each retriever: (1 / (k + rank_i(d)))
where k is a constant (default 60 in Elasticsearch) and rank_i(d) is the document's rank in retriever i's results. Documents that appear near the top of both result sets get the highest fused score.
Why Hybrid Search Matters for RAG
In a RAG pipeline, the retriever feeds context to the generator (LLM). If the retriever misses the most relevant document or returns irrelevant ones, the LLM will either hallucinate or produce a poor answer. Hybrid search improves RAG in several concrete ways:
- Higher recall: Combining two retrievers with different matching strategies surfaces relevant documents that either one alone would miss.
- Better precision on exact matches: BM25 ensures that queries containing product names, error codes, or technical identifiers still hit the right documents.
- Better precision on conceptual queries: Vector search ensures that natural-language questions like "how do I reset my password" match documents about "credential recovery."
- Robustness: You don't have to guess whether a query is keyword-heavy or semantic—the system handles both automatically.
Empirically, hybrid search with RRF consistently outperforms either BM25 or vector search alone on standard retrieval benchmarks, often by 5–15% in nDCG. For RAG, this translates directly to better-grounded, more accurate LLM responses.
Prerequisites and Setup
Before you begin, make sure you have the following:
- Elasticsearch 8.12 or later (RRF is GA from 8.8; the combined retrieval API is simplified in 8.12+).
- An embedding model. You can use Elasticsearch's built-in inference API with a hosted model, or generate embeddings externally with a model like OpenAI's
text-embedding-3-smallor a Hugging Face model. - Python 3.9+ with the
elasticsearchandopenaipackages installed.
pip install elasticsearch openai
Connect to your Elasticsearch cluster:
from elasticsearch import Elasticsearch
es = Elasticsearch(
"https://localhost:9200",
api_key="YOUR_API_KEY",
verify_certs=False # only for local dev
)
print(es.info())
Step 1: Create the Index Mapping
A hybrid search index needs two key fields: a text field for BM25 and a dense_vector field for kNN search. You'll also want metadata fields for filtering and display.
index_name = "rag-hybrid-docs"
mapping = {
"mappings": {
"properties": {
"title": {
"type": "text",
"analyzer": "standard"
},
"content": {
"type": "text",
"analyzer": "standard"
},
"content_embedding": {
"type": "dense_vector",
"dims": 1536,
"index": true,
"similarity": "cosine"
},
"category": {
"type": "keyword"
},
"source_url": {
"type": "keyword",
"index": false
}
}
}
}
if es.indices.exists(index=index_name):
es.indices.delete(index=index_name)
es.indices.create(index=index_name, body=mapping)
print(f"Index '{index_name}' created.")
Key points about the mapping:
- The
contentfield is a standardtextfield, which Elasticsearch indexes with BM25 by default. This powers the lexical half of hybrid search. - The
content_embeddingfield is adense_vectorwithindex: true, enabling approximate kNN search using HNSW. Setdimsto match your embedding model's output dimension (1536 for OpenAItext-embedding-3-small). similarity: "cosine"is the standard choice for normalized text embeddings.
Step 2: Ingest Documents with Embeddings
For each document, you store both the raw text (for BM25) and its embedding (for vector search). Here's an ingestion function that uses OpenAI to generate embeddings:
import openai
openai.api_key = "YOUR_OPENAI_API_KEY"
def get_embedding(text: str) -> list:
response = openai.embeddings.create(
input=text,
model="text-embedding-3-small"
)
return response.data[0].embedding
documents = [
{
"title": "Password Reset Guide",
"content": "To reset your password, click the 'Forgot Password' link on the login page. You will receive an email with a reset link valid for 30 minutes.",
"category": "account",
"source_url": "https://docs.example.com/auth/reset"
},
{
"title": "API Rate Limits",
"content": "Free tier accounts are limited to 100 requests per minute. Pro tier accounts allow 1000 requests per minute. Contact support to request a custom limit.",
"category": "api",
"source_url": "https://docs.example.com/api/limits"
},
{
"title": "Credential Recovery",
"content": "If you lose access to your account, use the credential recovery flow. Verify your identity via SMS or backup codes, then set a new password.",
"category": "account",
"source_url": "https://docs.example.com/auth/recovery"
},
{
"title": "Billing and Invoices",
"content": "Invoices are generated on the first of each month and can be downloaded from the billing dashboard. Payment methods include credit card and bank transfer.",
"category": "billing",
"source_url": "https://docs.example.com/billing/invoices"
}
]
for doc in documents:
doc["content_embedding"] = get_embedding(doc["content"])
es.index(index=index_name, document=doc)
es.indices.refresh(index=index_name)
print(f"Indexed {len(documents)} documents.")
Notice that we embed the content field and store the vector in content_embedding. In production, you'd batch the embedding API calls and use the Elasticsearch bulk API for throughput.
Step 3: Perform Hybrid Search with RRF
Now for the core of the tutorial. Elasticsearch 8.12+ supports a clean combined retrieval query where you specify both a text retriever (BM25) and a knn retriever, and RRF fuses them automatically.
def hybrid_search(query_text: str, top_k: int = 5):
query_embedding = get_embedding(query_text)
body = {
"retriever": {
"rrf": {
"retrievers": [
{
"standard": {
"query": {
"multi_match": {
"query": query_text,
"fields": ["title^2", "content"]
}
}
}
},
{
"knn": {
"field": "content_embedding",
"query_vector": query_embedding,
"k": 50,
"num_candidates": 100
}
}
],
"rank_window_size": 50,
"rank_constant": 60
}
},
"size": top_k,
"_source": ["title", "content", "category", "source_url"]
}
response = es.search(index=index_name, body=body)
return response["hits"]["hits"]
results = hybrid_search("I can't log into my account, how do I recover access?")
for hit in results:
score = hit["_score"]
src = hit["_source"]
print(f"[{score:.4f}] {src['title']} — {src['source_url']}")
print(f" {src['content'][:120]}...")
print()
Let's break down the query:
retriever.rrf: The top-level retriever is an RRF retriever that wraps two sub-retrievers.- Standard (BM25) retriever: Runs a
multi_matchquery againsttitle(boosted 2x) andcontent. This is the lexical half. - kNN retriever: Searches the
content_embeddingfield using the query's embedding.kis the number of nearest neighbors to return, andnum_candidatescontrols the HNSW candidate pool (higher = more accurate, slower). rank_window_size: How many top results from each retriever are considered for fusion (default 100).rank_constant: Thekin the RRF formula. Lower values give more weight to top ranks. The default of 60 works well in most cases.
For the query "I can't log into my account, how do I recover access?", the BM25 retriever will strongly match "Credential Recovery" (term overlap on "recover"), while the kNN retriever will match both "Password Reset Guide" and "Credential Recovery" (semantic similarity). RRF fuses these, typically surfacing both account-recovery documents at the top.
Adding Filters to Hybrid Search
You can apply filters that affect both retrievers using the filter clause inside each retriever, or use a pre-filter at the RRF level. Here's an example restricting results to the account category:
def hybrid_search_filtered(query_text: str, category: str, top_k: int = 5):
query_embedding = get_embedding(query_text)
category_filter = {"term": {"category": category}}
body = {
"retriever": {
"rrf": {
"retrievers": [
{
"standard": {
"query": {
"multi_match": {
"query": query_text,
"fields": ["title^2", "content"]
}
},
"filter": category_filter
}
},
{
"knn": {
"field": "content_embedding",
"query_vector": query_embedding,
"k": 50,
"num_candidates": 100,
"filter": category_filter
}
}
],
"rank_window_size": 50
}
},
"size": top_k,
"_source": ["title", "content", "category", "source_url"]
}
response = es.search(index=index_name, body=body)
return response["hits"]["hits"]
results = hybrid_search_filtered("how do I get back into my account", "account")
for hit in results:
print(f"[{hit['_score']:.4f}] {hit['_source']['title']}")
Applying the filter inside both retrievers ensures that the filtered candidate set is used before fusion, which is more accurate than filtering after fusion.
Step 4: Integrate Hybrid Search with an LLM for RAG
Now that you have high-quality retrieval, the final step is to feed the results into an LLM as context. Here's a complete RAG function:
def rag_answer(question: str, top_k: int = 5) -> str:
# 1. Retrieve relevant documents via hybrid search
results = hybrid_search(question, top_k=top_k)
# 2. Build the context string from retrieved documents
context_parts = []
for i, hit in enumerate(results, 1):
src = hit["_source"]
context_parts.append(
f"[{i}] Title: {src['title']}\n"
f" Source: {src['source_url']}\n"
f" Content: {src['content']}"
)
context = "\n\n".join(context_parts)
# 3. Construct the RAG prompt
system_prompt = (
"You are a helpful support assistant. Answer the user's question "
"using only the provided context. If the context does not contain "
"the answer, say you don't know. Cite sources by their number."
)
user_prompt = f"Context:\n{context}\n\nQuestion: {question}\n\nAnswer:"
# 4. Call the LLM
response = openai.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_prompt}
],
temperature=0.2
)
return response.choices[0].message.content
answer = rag_answer("I can't log into my account, how do I recover access?")
print(answer)
A typical response would synthesize information from both the "Password Reset Guide" and "Credential Recovery" documents, citing them as [1] and [2], because hybrid search surfaced both. With BM25 alone, you might only get the password reset doc; with vector search alone, you might get the recovery doc but miss the specific "30-minute link expiry" detail that BM25's term matching captures well.
Best Practices
1. Tune num_candidates and k for kNN
The num_candidates parameter controls how many HNSW nodes are explored per shard. Higher values improve recall at the cost of latency. For most RAG workloads, num_candidates between 100 and 500 and k between 50 and 100 is a good starting point. Benchmark on your own data using a labeled evaluation set.
2. Embed the Query the Same Way You Embed Documents
If you used text-embedding-3-small for ingestion, you must use the same model for query embeddings. Any mismatch in model, dimension, or normalization will silently degrade vector search quality.
3. Use Field Boosting in the BM25 Retriever
Titles are often highly informative. Boosting the title field (e.g., title^2 or title^3) in the BM25 query improves precision when the user's query closely matches a document title. Tune the boost factor on an evaluation set.
4. Evaluate with a Golden Dataset
Build a small set of (query, relevant_doc_id) pairs and measure recall@k and nDCG for BM25-only, vector-only, and hybrid configurations. This is the only reliable way to confirm that hybrid search is helping and to tune parameters like rank_constant and rank_window_size.
5. Consider Reranking for Final Precision
For maximum quality, add a cross-encoder reranker after hybrid retrieval. Retrieve a larger candidate set (e.g., top 50) with RRF, then rerank the top candidates with a more expensive but more accurate model. Elasticsearch supports this via learning-to-rank plugins or external reranker APIs.
6. Chunk Your Documents Appropriately
Hybrid search works best when each indexed unit is a coherent chunk (typically 200–800 tokens), not an entire long document. Chunking improves both BM25 precision (shorter fields have sharper term statistics) and vector search quality (embeddings of focused chunks are more semantically precise). Store parent document metadata so you can expand context after retrieval if needed.
7. Monitor and Reindex
Track query latency percentiles, recall metrics, and result click-through (if available). As your corpus grows, periodically reindex to rebuild HNSW graphs with optimal parameters, and re-embed documents if you upgrade your embedding model.
Conclusion
Hybrid search in Elasticsearch is a powerful, production-ready technique for improving retrieval quality in RAG systems. By combining BM25's precision on exact terms with dense vector search's semantic understanding, and fusing them with Reciprocal Rank Fusion, you get a retriever that is robust across a wide range of query types. The implementation is straightforward: create an index with both text and dense_vector fields, ingest documents with embeddings, and query using the RRF retriever API. By following the best practices around parameter tuning, evaluation, chunking, and optional reranking, you can build a RAG pipeline that consistently surfaces the right context for your LLM—leading to more accurate, trustworthy answers.