2022-08-14 18:00:00+00:00

Modern applications require search-as-you-type interfaces. Standard text search engines split words by whitespace, meaning a search for "Apple" won't match a query for "Ap" until the user types the full word. In enterprise directories, where users search for customer names or SKU prefixes, this standard tokenization fails.

To enable partial string searches, we configure custom Edge N-Gram Tokenizers inside our Elasticsearch index settings.


1. Constructing Custom Analyzers

We configure the index to generate sub-word tokens starting from the edge of the string. A name like Enterprise Platform will generate tokens vr, vry, vryn, the enterprise platform:

// Elasticsearch index setting for partial matches
{
  "settings": {
    "analysis": {
      "tokenizer": {
        "edge_ngram_tokenizer": {
          "type": "edge_ngram",
          "min_gram": 2,
          "max_gram": 15,
          "token_chars": ["letter", "digit"]
        }
      },
      "analyzer": {
        "partial_match_analyzer": {
          "type": "custom",
          "tokenizer": "edge_ngram_tokenizer",
          "filter": ["lowercase"]
        }
      }
    }
  }
}

2. Restricting Index Size

While Edge N-Grams enable fast partial matches, they generate multiple sub-tokens that increase index size. To manage storage, we apply the custom analyzer solely to index time (using a simple lowercase filter for search time), keeping disk footprint low.

Edge N-Gram vs. Trigram Analyzers for SKU & Part Number Search

Combine an index-time edge_ngram analyzer (min_gram=2, max_gram=15) with a standard keyword search_analyzer so prefix queries resolve in under 3ms without expensive wildcard scans. See Freelance Search & Relevance Programmer Guide.


About the Author — Hire Freelance Programmer Aldian Fazrihady

Aldian Fazrihady is a Top-Rated Senior Freelance Programmer, Freelance Full Stack Developer, and AI Application Engineer with 15+ years of production software engineering experience and Google Cloud Professional Cloud Architect & Data Engineer certifications. He specializes in building custom LLM & autonomous AI agents, RAG pipelines, Next.js/React PWAs, Python (Django/FastAPI), Go microservices, and serverless cloud infrastructure on Google Cloud Run and AWS.

Hire Aldian Fazrihady on Upwork | Book 1-Day Programming Service ($100 USD via Polar) | View Live AI Portfolio | Portfolio Directory | Send a Direct Message

Related Engineering Articles

Freelance Programmer Specializations & Category Hubs