Feature #7: Find Searching Time

Description

A typical search engine consists of many services, such as crawling, indexing, word stemming, synonym fetching, etc. When a user’s search query arrives at a web front end, it is dispatched to the above-mentioned search-related services. A single search query turns into a large number of calls to these services. Some of these calls are recursive, too. For instance, the stemming service may stem one or more words from the search query before invoking itself recursively on the rest of the search string.

Optimization is an ongoing effort and search engines keep trying to reduce the time it takes for the search results to be displayed on the user’s screen. To better guide this optimization activity, we instrumented code to achieve some statistics. Using these statistics, we want to figure out the time spent in each of the search-related services over a specific time window.

To find the time spent by each service, we will use log messages. Each service is identified in the logs by a service_id, and the logs contain the start and end timestamps of each invocation of the service. The logs will be stored in a list of strings, where each string will have the following format: "{service_id}:{"start" | "end"}:{timestamp}".

Below is an illustration of this format: