Introduction to the Search Service

Introduction

These days, most websites and applications provide search functionalities due to a number of advantages the search feature brings. For example, GitHub uses the search API to efficiently find a specific file or repository from more than 200 million repositories.

The search API takes a search query from a user and searches for relevant records based on the parameters in the query, as depicted in the following figure. In return, the user may get varied responses, including images, thumbnails of videos, recommendations, sponsored or featured advertisements, and primarily text results.

The search API can perform multiple operations on the matched records, such as limiting or sorting the number of records to return. In order to design the API for the search service, it’s important to understand the internal workings of the service. This background knowledge will help us optimally design different aspects of the API, such as communication protocols between the client and the server, middleware, and back-end services.

Let's discuss how the search service performs different functionalities.

How does the search service work?

To understand how the search system works, let's take a look at the following illustration that consists of an indexer and searcher.

When the search service receives a query, it is forwarded to the searcher. The searcher parses the query and looks into the index table to find the matching results. The index table consists of the terms and their mappings, which are created by the indexer. The responsibility of the indexer is to pre-index data as soon as it’s posted so that the searcher can quickly respond to search queries. The searcher aggregates all the results corresponding to the query and returns them to the search service. The following table describes the responsibilities of each component in the search service.

In response to a search query, the number of records can vary from zero to hundreds of thousands. Because of this, the data payload and time taken could be very large. Consequently, a limiting factor on the number of records to present to the user is required. This avoids overwhelming the client and burdening the back-end servers. For this purpose, we’ll use the pagination ...