← Back to DevBytes
Developer Tutorial July 07, 2026 16 min read

Designing a Search Engine with Event-Driven Architecture

What Is Event-Driven Search Engine Architecture?

An event-driven search engine replaces traditional synchronous request-response indexing with an architecture where every state change—document ingestion, updates, deletions, schema changes, and even query-side caching—propagates as discrete, immutable events. Rather than directly calling an indexing API and waiting for confirmation, producers emit events onto a durable event bus. Downstream consumers (indexing workers, analytics services, cache invalidators) react to these events asynchronously, each maintaining its own projection of the search corpus.

At its core, this design decouples the write path (document ingestion) from the read path (query serving). The search index becomes a materialized view derived from an append-only event log. This means you can rebuild your entire index from scratch by replaying the event history, run multiple indexing pipelines in parallel for A/B testing, or fan out indexing to heterogeneous backends—all without modifying the ingestion source.

Why Event-Driven Architecture Matters for Search

🚀 Deploy your AI agent in 10 minutes

Managed Hermes hosting. Zero DevOps. 100M tokens/mo included.

Try it free →

Traditional search indexing typically uses direct API calls to services like Elasticsearch or Solr. While straightforward, this approach introduces tight coupling, fragility under load, and difficult recovery scenarios. Event-driven architecture addresses these fundamental problems:

Core Components of an Event-Driven Search Engine

Before diving into code, let's map out the essential building blocks. A well-designed event-driven search engine comprises four primary components that work together through well-defined event contracts.

1. The Event Bus (Durable Log)

The event bus is the central nervous system. It must provide durable, ordered storage with the ability to replay events from arbitrary offsets. Apache Kafka is the canonical choice, but cloud-native alternatives like AWS Kinesis, Google Pub/Sub, or Redpanda serve the same role. The key property is durability: once an event is acknowledged, it must survive consumer failures and be retrievable hours or days later for recovery scenarios.

2. The Ingestion Gateway

A lightweight HTTP or gRPC service that validates incoming documents, assigns unique identifiers, and publishes DocumentIndexed events to the bus. This service never touches the search index directly—it is purely an event producer. It should be designed for high throughput and minimal latency, returning success to the caller once the event is safely on the bus.

3. Indexing Workers (Event Consumers)

These are the workhorses that read events from the bus, transform document payloads into the inverted index structures required by the search backend, and flush them efficiently. Workers can be scaled independently based on indexing lag. They also handle DocumentDeleted and DocumentUpdated events, maintaining the materialized view.

4. Query Service

The query service fronts the search backend and handles user search requests. It is not event-driven in its read path (search queries remain synchronous request-response), but it consumes configuration events like schema changes or relevancy tuning updates. The query service may also consume a subset of events for real-time features like "search-as-you-type" on recently added documents.

Event Schema Design

Consistent, evolvable event schemas are critical. Each event should carry enough information for consumers to act without calling back to the source system. Here is a recommended event envelope and payload structure using a schema registry approach:

// Event envelope — every event published to the bus follows this structure
{
  "eventId": "evt_9a7f3c1d-4b2e-4a8a-9e5f-1d6c8b3a2e0f",
  "eventType": "DocumentIndexed",
  "timestamp": "2025-03-15T10:30:00.000Z",
  "version": "2",
  "traceId": "trace_abc123",
  "payload": {
    // Event-specific data goes here
  }
}

// DocumentIndexed event payload — version 2
{
  "documentId": "doc_42",
  "indexName": "products",
  "content": {
    "title": "Wireless Noise-Cancelling Headphones",
    "description": "Premium over-ear headphones with 30-hour battery life...",
    "price": 249.99,
    "category": "electronics",
    "tags": ["audio", "wireless", "bluetooth", "noise-cancelling"]
  },
  "metadata": {
    "source": "catalog-service",
    "locale": "en_US",
    "checksum": "sha256:a1b2c3..."
  },
  "operation": "UPSERT"
}

// DocumentDeleted event payload
{
  "documentId": "doc_42",
  "indexName": "products",
  "reason": "product_discontinued",
  "deletedAt": "2025-03-20T14:00:00.000Z"
}

How to Build It: Step-by-Step Implementation

Let's build a working event-driven search engine using Node.js, Express for the ingestion gateway, a simulated event bus (which you can swap for Kafka in production), Elasticsearch as the search backend, and a modular indexing worker. Every code block is self-contained and ready to adapt.

Step 1: Define the Event Bus Abstraction

Start with a clean event bus interface. In production, this would wrap a Kafka producer/consumer. For clarity, we build an in-memory bus that honors the same contract—publish, subscribe with offsets, and replay capability. This lets you test the architecture without external dependencies.

// eventBus.ts — Abstract event bus with publish/subscribe and replay semantics

export interface EventEnvelope {
  eventId: string;
  eventType: string;
  timestamp: string;
  version: string;
  traceId: string;
  payload: Record;
}

export type EventHandler = (event: EventEnvelope) => Promise;

export interface EventBus {
  publish(topic: string, event: EventEnvelope): Promise;
  subscribe(topic: string, handler: EventHandler, options?: {
    fromOffset?: number;
    groupId?: string;
  }): Promise;
  getEvents(topic: string, fromOffset: number, limit: number): Promise;
}

// In-memory implementation for development and testing
export class InMemoryEventBus implements EventBus {
  private topics: Map = new Map();
  private subscribers: Map> = new Map();
  private offsets: Map = new Map();

  async publish(topic: string, event: EventEnvelope): Promise {
    if (!this.topics.has(topic)) {
      this.topics.set(topic, []);
    }
    const events = this.topics.get(topic)!;
    events.push(event);

    // Notify subscribers asynchronously
    const topicSubscribers = this.subscribers.get(topic);
    if (topicSubscribers) {
      for (const [, handlers] of topicSubscribers) {
        for (const handler of handlers) {
          // Fire-and-forget per handler (caught internally)
          handler(event).catch(err =>
            console.error(`Handler error for event ${event.eventId}:`, err)
          );
        }
      }
    }
  }

  async subscribe(
    topic: string,
    handler: EventHandler,
    options?: { fromOffset?: number; groupId?: string }
  ): Promise {
    const groupId = options?.groupId || 'default';
    if (!this.subscribers.has(topic)) {
      this.subscribers.set(topic, new Map());
    }
    const topicSubs = this.subscribers.get(topic)!;
    if (!topicSubs.has(groupId)) {
      topicSubs.set(groupId, []);
    }
    topicSubs.get(groupId)!.push(handler);

    // Replay existing events from the requested offset
    const currentOffset = this.offsets.get(`${topic}:${groupId}`) || 0;
    const startOffset = options?.fromOffset ?? currentOffset;
    const events = this.topics.get(topic) || [];

    for (let i = startOffset; i < events.length; i++) {
      await handler(events[i]);
    }
    this.offsets.set(`${topic}:${groupId}`, events.length);
  }

  async getEvents(topic: string, fromOffset: number, limit: number): Promise {
    const events = this.topics.get(topic) || [];
    return events.slice(fromOffset, fromOffset + limit);
  }
}

Step 2: Create the Event Factory

Every event needs a consistent structure. A factory function ensures all required envelope fields are populated, generates unique IDs, and attaches tracing information.

// eventFactory.ts — Standardized event creation

import { randomUUID } from 'crypto';
import { EventEnvelope } from './eventBus';

export function createEvent(
  eventType: string,
  payload: Record,
  traceId?: string
): EventEnvelope {
  return {
    eventId: `evt_${randomUUID()}`,
    eventType,
    timestamp: new Date().toISOString(),
    version: '1',
    traceId: traceId || `trace_${randomUUID().slice(0, 8)}`,
    payload,
  };
}

// Specific event creators for the search domain
export function documentIndexedEvent(
  documentId: string,
  indexName: string,
  content: Record,
  metadata?: Record
): EventEnvelope {
  return createEvent('DocumentIndexed', {
    documentId,
    indexName,
    content,
    metadata: metadata || {},
    operation: 'UPSERT',
  });
}

export function documentDeletedEvent(
  documentId: string,
  indexName: string,
  reason?: string
): EventEnvelope {
  return createEvent('DocumentDeleted', {
    documentId,
    indexName,
    reason: reason || 'user_requested',
    deletedAt: new Date().toISOString(),
  });
}

export function indexRebuildRequestedEvent(
  indexName: string,
  fromOffset: number
): EventEnvelope {
  return createEvent('IndexRebuildRequested', {
    indexName,
    fromOffset,
    requestedAt: new Date().toISOString(),
  });
}

Step 3: Build the Ingestion Gateway

The ingestion gateway is a simple Express server that accepts document submissions, validates them, and publishes events. It never directly calls Elasticsearch—this is the crucial architectural difference.

// ingestionGateway.ts — HTTP API that publishes events, never touches the index

import express, { Request, Response } from 'express';
import { EventBus, EventEnvelope } from './eventBus';
import { documentIndexedEvent, documentDeletedEvent } from './eventFactory';

const app = express();
app.use(express.json());

// Simple in-memory document store for validation (production would use a DB)
const documentExists = new Map();

export function createIngestionGateway(eventBus: EventBus) {
  // POST /documents — Index or update a document
  app.post('/documents', async (req: Request, res: Response) => {
    try {
      const { documentId, indexName, content, metadata } = req.body;

      // Validate required fields
      if (!documentId || !indexName || !content) {
        return res.status(400).json({
          error: 'Missing required fields: documentId, indexName, content',
        });
      }

      if (typeof content !== 'object' || Array.isArray(content)) {
        return res.status(400).json({
          error: 'content must be a JSON object',
        });
      }

      // Build and publish the event
      const event = documentIndexedEvent(
        documentId,
        indexName,
        content,
        metadata || {}
      );

      await eventBus.publish('search-events', event);
      documentExists.set(documentId, true);

      console.log(`Published DocumentIndexed event: ${event.eventId}`);

      // Return immediately — indexing happens asynchronously
      res.status(202).json({
        status: 'accepted',
        eventId: event.eventId,
        documentId,
        message: 'Document queued for indexing',
      });
    } catch (error) {
      console.error('Error publishing event:', error);
      res.status(500).json({ error: 'Internal server error' });
    }
  });

  // DELETE /documents/:documentId — Remove a document from the index
  app.delete('/documents/:documentId', async (req: Request, res: Response) => {
    try {
      const { documentId } = req.params;
      const { indexName } = req.query;

      if (!indexName || typeof indexName !== 'string') {
        return res.status(400).json({
          error: 'Query parameter indexName is required',
        });
      }

      const event = documentDeletedEvent(documentId, indexName, 'api_request');
      await eventBus.publish('search-events', event);
      documentExists.set(documentId, false);

      res.status(202).json({
        status: 'accepted',
        eventId: event.eventId,
        documentId,
        message: 'Document deletion queued',
      });
    } catch (error) {
      console.error('Error publishing delete event:', error);
      res.status(500).json({ error: 'Internal server error' });
    }
  });

  // GET /health — Health check
  app.get('/health', (_req: Request, res: Response) => {
    res.json({ status: 'healthy', timestamp: new Date().toISOString() });
  });

  return app;
}

Step 4: Implement the Indexing Worker

The indexing worker subscribes to search-events and maintains an Elasticsearch index. It buffers events and flushes in bulk for efficiency. The worker also handles rebuild requests by replaying from a specified offset.

// indexingWorker.ts — Consumes events and maintains the search index

import { EventBus, EventEnvelope } from './eventBus';

// Abstract search backend interface — swap Elasticsearch, Solr, Meilisearch, etc.
interface SearchBackend {
  indexDocument(
    indexName: string,
    documentId: string,
    content: Record
  ): Promise;
  deleteDocument(indexName: string, documentId: string): Promise;
  bulkIndex(
    indexName: string,
    documents: Array<{ id: string; content: Record }>
  ): Promise;
  bulkDelete(
    indexName: string,
    documentIds: string[]
  ): Promise;
}

// Simulated search backend for development (replace with real Elasticsearch client)
class InMemorySearchBackend implements SearchBackend {
  private indexes: Map>> = new Map();

  async indexDocument(
    indexName: string,
    documentId: string,
    content: Record
  ): Promise {
    if (!this.indexes.has(indexName)) {
      this.indexes.set(indexName, new Map());
    }
    this.indexes.get(indexName)!.set(documentId, content);
    console.log(`Indexed doc ${documentId} in ${indexName}`);
  }

  async deleteDocument(indexName: string, documentId: string): Promise {
    this.indexes.get(indexName)?.delete(documentId);
    console.log(`Deleted doc ${documentId} from ${indexName}`);
  }

  async bulkIndex(
    indexName: string,
    documents: Array<{ id: string; content: Record }>
  ): Promise {
    if (!this.indexes.has(indexName)) {
      this.indexes.set(indexName, new Map());
    }
    const index = this.indexes.get(indexName)!;
    for (const doc of documents) {
      index.set(doc.id, doc.content);
    }
    console.log(`Bulk indexed ${documents.length} documents in ${indexName}`);
  }

  async bulkDelete(indexName: string, documentIds: string[]): Promise {
    const index = this.indexes.get(indexName);
    if (index) {
      for (const id of documentIds) {
        index.delete(id);
      }
    }
    console.log(`Bulk deleted ${documentIds.length} documents from ${indexName}`);
  }

  // Query helper for testing
  search(indexName: string, query: string): Array<{ id: string; content: Record }> {
    const index = this.indexes.get(indexName);
    if (!index) return [];
    const results: Array<{ id: string; content: Record }> = [];
    const lowerQuery = query.toLowerCase();
    for (const [id, content] of index) {
      const searchableText = JSON.stringify(content).toLowerCase();
      if (searchableText.includes(lowerQuery)) {
        results.push({ id, content });
      }
    }
    return results;
  }
}

// Indexing worker configuration
interface WorkerConfig {
  flushIntervalMs: number;
  maxBatchSize: number;
  indexName: string;
}

export class IndexingWorker {
  private buffer: Array<{ action: 'index' | 'delete'; event: EventEnvelope }> = [];
  private flushTimer: NodeJS.Timeout | null = null;
  private backend: SearchBackend;
  private bus: EventBus;
  private config: WorkerConfig;

  constructor(bus: EventBus, backend: SearchBackend, config: WorkerConfig) {
    this.bus = bus;
    this.backend = backend;
    this.config = config;
  }

  async start(): Promise {
    // Subscribe to the main search events topic
    await this.bus.subscribe('search-events', this.handleEvent.bind(this), {
      groupId: 'indexing-workers',
    });

    // Also subscribe to rebuild requests
    await this.bus.subscribe('index-admin', this.handleAdminEvent.bind(this), {
      groupId: 'indexing-workers',
    });

    // Start periodic flush
    this.flushTimer = setInterval(() => {
      this.flushBuffer();
    }, this.config.flushIntervalMs);

    console.log('Indexing worker started');
  }

  private async handleEvent(event: EventEnvelope): Promise {
    switch (event.eventType) {
      case 'DocumentIndexed':
        this.buffer.push({ action: 'index', event });
        break;
      case 'DocumentDeleted':
        this.buffer.push({ action: 'delete', event });
        break;
      default:
        console.log(`Skipping unknown event type: ${event.eventType}`);
        return;
    }

    // Flush immediately if batch size exceeded
    if (this.buffer.length >= this.config.maxBatchSize) {
      await this.flushBuffer();
    }
  }

  private async handleAdminEvent(event: EventEnvelope): Promise {
    if (event.eventType === 'IndexRebuildRequested') {
      const { indexName, fromOffset } = event.payload as {
        indexName: string;
        fromOffset: number;
      };
      console.log(`Rebuild requested for ${indexName} from offset ${fromOffset}`);
      await this.rebuildIndex(indexName, fromOffset);
    }
  }

  async flushBuffer(): Promise {
    if (this.buffer.length === 0) return;

    const batch = [...this.buffer];
    this.buffer = [];

    const indexDocs: Array<{ id: string; content: Record }> = [];
    const deleteIds: string[] = [];
    const indexName = this.config.indexName;

    for (const item of batch) {
      if (item.action === 'index') {
        const p = item.event.payload as {
          documentId: string;
          content: Record;
        };
        indexDocs.push({ id: p.documentId, content: p.content });
      } else if (item.action === 'delete') {
        const p = item.event.payload as { documentId: string };
        deleteIds.push(p.documentId);
      }
    }

    try {
      if (indexDocs.length > 0) {
        await this.backend.bulkIndex(indexName, indexDocs);
      }
      if (deleteIds.length > 0) {
        await this.backend.bulkDelete(indexName, deleteIds);
      }
      console.log(
        `Flushed ${indexDocs.length} indexes, ${deleteIds.length} deletes to ${indexName}`
      );
    } catch (error) {
      console.error('Flush failed, requeuing batch:', error);
      // Requeue failed items back into buffer for retry
      this.buffer.unshift(...batch);
    }
  }

  async rebuildIndex(indexName: string, fromOffset: number): Promise {
    console.log(`Starting rebuild of ${indexName} from offset ${fromOffset}`);
    // In a real system, you'd create a new index, replay all events,
    // then atomically swap the alias. Here we replay into the same backend.
    const events = await this.bus.getEvents('search-events', fromOffset, 1000);

    for (const event of events) {
      if (event.eventType === 'DocumentIndexed') {
        const p = event.payload as {
          documentId: string;
          content: Record;
        };
        await this.backend.indexDocument(indexName, p.documentId, p.content);
      } else if (event.eventType === 'DocumentDeleted') {
        const p = event.payload as { documentId: string };
        await this.backend.deleteDocument(indexName, p.documentId);
      }
    }
    console.log(`Rebuild complete for ${indexName}, processed ${events.length} events`);
  }

  async stop(): Promise {
    if (this.flushTimer) {
      clearInterval(this.flushTimer);
    }
    await this.flushBuffer();
  }
}

Step 5: Build the Query Service

The query service handles search requests synchronously against the search backend. It also consumes configuration events for dynamic index settings and can provide near-real-time access to recently indexed documents by tailing the event stream.

// queryService.ts — Read-side service for search queries

import express, { Request, Response } from 'express';
import { EventBus, EventEnvelope } from './eventBus';
import { SearchBackend } from './indexingWorker'; // Reuse the interface

const app = express();
app.use(express.json());

export function createQueryService(
  backend: SearchBackend & { search: (indexName: string, query: string) => Array<{ id: string; content: Record }> },
  eventBus: EventBus
) {
  // GET /search?q=...&index=... — Main search endpoint
  app.get('/search', async (req: Request, res: Response) => {
    try {
      const query = req.query.q as string;
      const indexName = (req.query.index as string) || 'default';

      if (!query || query.trim().length === 0) {
        return res.status(400).json({ error: 'Query parameter "q" is required' });
      }

      const results = backend.search(indexName, query);

      res.json({
        query,
        indexName,
        totalResults: results.length,
        results: results.slice(0, 20), // Paginate in production
        searchedAt: new Date().toISOString(),
      });
    } catch (error) {
      console.error('Search error:', error);
      res.status(500).json({ error: 'Search failed' });
    }
  });

  // GET /documents/:documentId — Direct document lookup
  app.get('/documents/:documentId', async (req: Request, res: Response) => {
    try {
      const { documentId } = req.params;
      const indexName = (req.query.index as string) || 'default';

      // In our in-memory backend, search with the exact ID
      const results = backend.search(indexName, documentId);
      const doc = results.find(r => r.id === documentId);

      if (!doc) {
        return res.status(404).json({ error: 'Document not found' });
      }

      res.json({ documentId, indexName, content: doc.content });
    } catch (error) {
      console.error('Document lookup error:', error);
      res.status(500).json({ error: 'Lookup failed' });
    }
  });

  // Subscribe to index configuration events (e.g., relevancy tuning)
  eventBus.subscribe('index-config', async (event: EventEnvelope) => {
    console.log(`Received config event: ${event.eventType}`, event.payload);
    // In production: update query parsers, synonym maps, boost configurations
  }, { groupId: 'query-service' });

  return app;
}

Step 6: Wire Everything Together in the Main Application

This final piece assembles all components, starts the services, and demonstrates the complete event flow from ingestion to query.

// main.ts — Application entry point that wires all components together

import { InMemoryEventBus } from './eventBus';
import { createIngestionGateway } from './ingestionGateway';
import { IndexingWorker, InMemorySearchBackend } from './indexingWorker';
import { createQueryService } from './queryService';
import { indexRebuildRequestedEvent } from './eventFactory';

async function main() {
  console.log('Starting Event-Driven Search Engine...\n');

  // 1. Initialize infrastructure
  const eventBus = new InMemoryEventBus();
  const searchBackend = new InMemorySearchBackend();

  // 2. Create services
  const ingestionApp = createIngestionGateway(eventBus);
  const queryApp = createQueryService(searchBackend, eventBus);

  // 3. Start indexing worker
  const worker = new IndexingWorker(eventBus, searchBackend, {
    flushIntervalMs: 5000,  // Flush every 5 seconds
    maxBatchSize: 50,       // Or when 50 events accumulate
    indexName: 'default',
  });
  await worker.start();

  // 4. Start HTTP servers
  const INGESTION_PORT = 3001;
  const QUERY_PORT = 3002;

  ingestionApp.listen(INGESTION_PORT, () => {
    console.log(`Ingestion API running on http://localhost:${INGESTION_PORT}`);
    console.log('POST /documents — Index a document');
    console.log('DELETE /documents/:id — Remove a document\n');
  });

  queryApp.listen(QUERY_PORT, () => {
    console.log(`Query API running on http://localhost:${QUERY_PORT}`);
    console.log('GET /search?q=...&index=... — Search documents');
    console.log('GET /documents/:id — Look up a document\n');
  });

  // 5. Demonstrate the event-driven flow with sample operations
  console.log('=== DEMONSTRATION: Ingesting sample documents ===\n');

  // Simulate HTTP POST requests by publishing events directly
  const { documentIndexedEvent: createDocEvent, documentDeletedEvent: createDeleteEvent } =
    await import('./eventFactory');

  // Index three sample documents
  await eventBus.publish('search-events', createDocEvent(
    'doc_1', 'default',
    { title: 'Introduction to Event-Driven Architecture', category: 'software', tags: ['events', 'architecture'] }
  ));
  await eventBus.publish('search-events', createDocEvent(
    'doc_2', 'default',
    { title: 'Building Scalable Search Engines', category: 'software', tags: ['search', 'scalability'] }
  ));
  await eventBus.publish('search-events', createDocEvent(
    'doc_3', 'default',
    { title: 'Kafka Streams for Real-Time Indexing', category: 'data', tags: ['kafka', 'streaming', 'indexing'] }
  ));

  console.log('Published 3 DocumentIndexed events');

  // Allow time for the worker to process and flush
  await sleep(6000);

  // 6. Demonstrate search
  console.log('\n=== SEARCH DEMONSTRATION ===');
  const results = searchBackend.search('default', 'search');
  console.log(`Search for "search" returned ${results.length} results:`);
  results.forEach(r => console.log(`  - [${r.id}] ${r.content.title}`));

  // 7. Demonstrate deletion
  console.log('\n=== DELETION DEMONSTRATION ===');
  await eventBus.publish('search-events', createDeleteEvent('doc_3', 'default', 'demo_cleanup'));
  console.log('Published DocumentDeleted event for doc_3');

  await sleep(6000);
  const resultsAfterDelete = searchBackend.search('default', 'kafka');
  console.log(`Search for "kafka" after deletion: ${resultsAfterDelete.length} results`);

  // 8. Demonstrate index rebuild
  console.log('\n=== REBUILD DEMONSTRATION ===');
  const rebuildEvent = indexRebuildRequestedEvent('default', 0);
  await eventBus.publish('index-admin', rebuildEvent);
  console.log('Published IndexRebuildRequested event');

  await sleep(2000);
  const resultsAfterRebuild = searchBackend.search('default', 'architecture');
  console.log(`Search for "architecture" after rebuild: ${resultsAfterRebuild.length} results`);

  console.log('\n✅ Event-driven search engine demonstration complete');
  console.log('The system is running — try these commands:');
  console.log('  curl -X POST http://localhost:3001/documents -H "Content-Type: application/json" -d \'{"documentId":"doc_100","indexName":"default","content":{"title":"Test"}}\'');
  console.log('  curl "http://localhost:3002/search?q=test&index=default"');

  // Graceful shutdown handler
  process.on('SIGINT', async () => {
    console.log('\nShutting down...');
    await worker.stop();
    process.exit(0);
  });
}

function sleep(ms: number): Promise {
  return new Promise(resolve => setTimeout(resolve, ms));
}

main().catch(console.error);

Step 7: Production-Grade Elasticsearch Worker (Real Backend Adapter)

Here is a production-ready adapter that replaces the in-memory backend with Elasticsearch. It demonstrates bulk indexing, retry logic, and alias-based index swapping for zero-downtime rebuilds.

// elasticsearchBackend.ts — Production Elasticsearch adapter

import { Client } from '@elastic/elasticsearch'; // Install: npm install @elastic/elasticsearch
import { SearchBackend } from './indexingWorker';

export class ElasticsearchBackend implements SearchBackend {
  private client: Client;
  private indexAlias: string;

  constructor(esUrl: string, indexAlias: string) {
    this.client = new Client({ node: esUrl });
    this.indexAlias = indexAlias;
  }

  private getWriteIndex(): string {
    // Use a dated index name for writing, alias for reading
    const today = new Date().toISOString().slice(0, 10).replace(/-/g, '');
    return `${this.indexAlias}_${today}`;
  }

  async indexDocument(
    indexName: string,
    documentId: string,
    content: Record
  ): Promise {
    const writeIndex = this.getWriteIndex();
    await this.client.index({
      index: writeIndex,
      id: documentId,
      body: content,
      refresh: false, // Don't refresh on every single doc
    });
  }

  async deleteDocument(indexName: string, documentId: string): Promise {
    const writeIndex = this.getWriteIndex();
    try {
      await this.client.delete({
        index: writeIndex,
        id: documentId,
      });
    } catch (error: any) {
      // Elasticsearch throws if document doesn't exist — that's okay
      if (error.meta?.

🚀 Need a reliable AI agent for your project?

Deploy Hermes Agent in 10 minutes. Managed hosting, zero DevOps.

Get Started — $23.99/mo
← Back to all articles