Design a Distributed Search

What is a search system?

Behind every search bar, there is a search system. A search system is a system that takes some text input, a search query, from the user and returns the relevant content in a few seconds or less. There are three main components of a search system, namely:

• A crawler, which fetches content and creates documentsFor a search engine, a document consists of the text extracted from a web page. In a movie store’s web page, a document could be a JSON object containing titles, descriptions, and other metadata of the videos upon which we want to perform search queries. The documents could be JSON or any other suitable format. Documents are stored on a distributed storage like S3 or HDFS..

• An indexer, which builds a searchable index.

• A searcher, which responds to search queries by running the search query on the index created by the indexer.