Semantic Search vs Keyword Search and Why You Need Both

If you search for “authentication” in a keyword-based system, it might miss documents that only mention “login security” because none of the exact words match. Semantic search has the opposite weakness. If you search for a specific identifier like the authentication error code ERROR_401_EXPIRED, it may prioritize broadly related passages about login failures while ranking the document containing that exact error code much lower.

A production search system needs both: keyword search for exact terms, semantic search for meaning, and a merge step that combines their rankings into one list. That combination is called hybrid search, and it’s become the standard retrieval strategy for RAG systems.

We’ll walk through how each approach works, where each one breaks down, and how hybrid search merges them.

What is Keyword Search?

Keyword search, also called lexical search, matches the words in a query against the words in documents. Most implementations use the BM25 scoring algorithm, which ranks documents higher when they contain the query terms more often, when those terms are rare across the entire collection, and when the documents aren’t padded with unrelated text. A common word like “revenue” appears in nearly every page of an earnings report, so a match carries little weight. A string like ERROR_401_EXPIRED appears in one runbook, so BM25 treats that match as a strong signal.

This mechanical behavior is the strength. Keyword search finds exact identifiers, error codes, product names and quoted phrases every time they appear. It runs fast, and its results are easy to explain to users and auditors: a document matched because it contains the words you typed.

What is Semantic Search?

Semantic search compares meaning instead of spelling. It uses an embedding model to convert text into a vector, where vectors representing related ideas are positioned close together. The query is converted into a vector as well, and the engine returns the documents whose vectors are nearest to it, usually measured by cosine similarity.

Vocabulary stops being a constraint. A search for “authentication” retrieves the “login security” document because the two phrases occupy the same semantic neighborhood. Users can ask full questions in plain language, like “How do I let users sign in with their Google account?”, and retrieve OAuth documentation that shares almost no words with the query.

Where Each Approach Breaks Down

The two engines fail in nearly opposite situations, which is what makes them such good partners.

Keyword search breaks on vocabulary mismatch. Writers and searchers rarely pick the same words. A support agent types “refund policy” while the document says “returns and reimbursements,” and BM25 scores it near zero. Keyword search also carries no model of intent, so it can’t tell that “How do I cancel my plan?” and “subscription termination steps” want the same answer.

Semantic search breaks on exact strings. Embeddings compress text into general meaning, and rare tokens lose their identity in that compression. Error codes, part numbers, acronyms and person or product names tend to retrieve similar-looking content instead of the exact match. A query for “OpenEdge 12.8 release notes” may return notes for a nearby version, since the two documents are semantically near twins.

The second failure mode is the one we see teams underestimate. Demos run on natural language questions, so semantic search looks flawless until a real user pastes an invoice number or an error code into the search bar.

What is Hybrid Search?

Hybrid search runs both engines on every query and merges the two ranked lists into one. The merge step needs care because BM25 scores and cosine similarities live on different scales, so comparing them directly is meaningless. Reciprocal Rank Fusion sidesteps the scale problem by using positions instead of scores. Each document earns points based on where it ranks in each list, and a document that ranks high in both lists accumulates the most points and floats to the top.

Most implementations expose a weight between the two signals. A support knowledge base full of error codes benefits from a keyword tilt, while a policy wiki that people query in plain language benefits from a semantic tilt. We’d start balanced and adjust only after watching real queries fail.

Why Hybrid Search Matters for RAG

A RAG pipeline can only generate answers from the passages retrieval hands it. If retrieval misses the relevant passage, the LLM never sees it and can’t cite it. No amount of prompt engineering or model quality can recover a fact that never entered the context.

Consider a question like “What was revenue guidance for FY 2026?” Semantic retrieval surfaces passages discussing financial outlook and guidance, while keyword retrieval ensures the exact phrase “FY 2026” is matched so the model cites the correct fiscal year instead of a semantically similar one. Combined, the two approaches retrieve both the relevant passage and the precise details the LLM needs to generate an accurate answer.

Hybrid Search in Progress Agentic RAG

The Progress Agentic RAG platform runs keyword and semantic search together by default. The platform is powered by NucliaDB, which unifies semantic search, keyword search, metadata search and knowledge graph traversal in a single store, so we never maintain a separate keyword index next to a vector database or write our own merging logic.

The Search configuration panel exposes Reciprocal Rank Fusion directly. We can adjust the weighting to boost semantic results over keyword results, or the reverse, depending on how our users search and the types of information they need to retrieve.

Wrap Up

Keyword search matches exact words with speed and predictability, semantic search matches meaning across different vocabulary. Each one fails where the other succeeds. Hybrid search runs both and merges their rankings with Reciprocal Rank Fusion, which is why it has become the default retrieval strategy for RAG. Progress Agentic RAG ships this behavior out of the box, with the fusion weighting a single setting away.

For more details and to get started with Progress Agentic RAG, be sure to check out the following resources:

Frequently Asked Questions

Is semantic search always better than keyword search?

No. Semantic search wins when queries are phrased differently from the documents, and keyword search wins when the query contains an exact string such as an error code, a product name, or a quoted phrase. Each covers the other’s blind spot, which is why production systems merge both.

What is Reciprocal Rank Fusion in hybrid search?

Reciprocal Rank Fusion (RRF) is the algorithm that merges the ranked lists from keyword and semantic search. Since the two engines score documents on incompatible scales, RRF ignores the raw scores and awards each document points based on its rank position in each list. Documents that rank well in both lists rise to the top of the fused ranking, and many systems, including Progress Agentic RAG, let us weight one list more heavily than the other.

Do I need a separate keyword index and vector database to run hybrid search?

It depends on the stack. Many teams pair a keyword engine like Elasticsearch with a vector database and merge results in application code, which works but doubles the infrastructure and keeps two indexes in sync. Unified stores such as NucliaDB index text for both search modes at ingestion, so hybrid search runs as a single query against one index.