Architectural Components

Architecture

Let’s have a look at the architectural components that are integral in creating the search engine.

Let’s briefly look at each component’s role in answering the searcher’s query.

Query rewriting

Queries are often poorly worded and far from describing the searcher’s actual information needs. Hence, we use query rewriting to increase recall, i.e., to retrieve a larger set of relevant results. Query rewriting has multiple components which are mentioned below.

Spell checker

Spell checking queries is an integral part of the search experience and is assumed to be a necessary feature of modern search. Spell checking allows you to fix basic spelling mistakes like “itlian restaurat” to “italian restaurant”.

Query expansion

Query expansion improves search result retrieval by adding terms to the user’s query. Essentially, these additional terms minimize the mismatch between the searcher’s query and available documents.

Hence, after correcting the spelling mistakes, we would want to expand terms, e.g., for the query “italian restaurant”, we should expand “restaurant” to food or recipe to look at all potential candidates (i.e., web pages) for this query.

📝 The reverse, i.e., query relaxation, serves the same purpose. For example, a search for “good italian restaurant” can be relaxed to “italian restaurant”.

Query understanding

This stage includes figuring out the main intent ...