Counting at Scale with HyperLogLog

HyperLogLog

In Redis, HyperLogLog is a probabilistic data structure that can be used to count the number of members (cardinality) of a set. It trades perfect accuracy for efficient space utilization, which makes it better than a set as the number of elements increases.

The following characteristics make HyperLogLog suitable for high volume operations:

• Memory usage: It uses a fixed amount of memory—less than 12 KB per key for up to 2⁶⁴ cardinalities.

• Performance: HyperLogLog commands have constant time complexity—O(1) per key.

• Accuracy trade-off: The cardinality returned is not exact, with a standard error of less than 1%—approximately 0.81% with default configuration.

Commands for HyperLogLog

To better understand how HyperLogLog works, let’s explore the commands associated with this data type.

The PFAdd command

Use the PFAdd command to add items to a HyperLogLog. In this example, we add user IDs to a HyperLogLog named views_from_user.

Behind the scenes, the cardinality of the set is calculated.