Avoiding Cognitive Biases

Cognitive biases are mental shortcuts that people use to make sense of the world around them. These biases can influence how people perceive and interpret information and can lead to inaccurate or distorted judgments. There are many different types of cognitive biases, and they can affect various aspects of decision-making.

In the context of setting product analytics, cognitive biases can play a role in how metrics and goals are chosen and how progress toward those goals is evaluated. For example, if a team is setting goals for a product and they are affected by confirmation bias, they may only consider information that supports their preconceived ideas about the product and ignore information that contradicts those ideas. This can lead to the selection of metrics that are not representative of the true success of the product and may not accurately reflect the needs of the users.

To avoid the influence of cognitive biases in setting product analytics, it’s important to be aware of these biases and take steps to mitigate their impact. This can include involving a diverse group of stakeholders in the process of setting metrics and goals, seeking out alternative viewpoints and perspectives, and using objective data to inform decision-making.

Being aware of cognitive biases can help product managers avoid making decisions that are based on distorted or incomplete information. It can also help them be more open to alternative viewpoints and perspectives and to consider a broader range of options when making decisions. Although there are over a hundred different types of cognitive biases, in the following section, we'll learn about the most common ones that we should keep in mind when setting and interpreting metrics across product usage, customer feedback, revenue, and so on.

Confirmation bias

Confirmation bias is the tendency to look for, understand, and choose the information that backs up what we already think or believe. This bias can affect API metrics by making it more likely that metrics are chosen that don’t show the real success of the API and may not accurately show what users want. The following illustration shows how our beliefs may not always overlap with data, and this leads to us ignoring evidence that doesn’t agree with our beliefs, resulting in confirmation bias.