Cognitive Bias Awareness and Countermeasures

Let's learn about some more types of cognitive biases.

Status quo bias

Status quo bias refers to the tendency to prefer the status quo and to be resistant to change. In the context of analyzing product metrics, status quo bias can occur if we are unwilling to consider alternative explanations or viewpoints, or if we are resistant to making changes based on data. The following illustration shows how status quo bias can lead to tunnel vision, where we explore solutions based on known variables, blinding us to solutions that lie outside of the status quo.

Imagine that a product manager is responsible for managing a software application that includes a GraphQL API that allows developers to query and modify data stored in the application. The API has been available for several years, and many developers have built integrations that rely on the existing API schema and functionality.

However, the product manager has received feedback from some developers that the API schema is difficult to understand and that certain parts of it are not well documented. They have suggested that adding more documentation and clarifying the schema could make the API more useful and easier to work with.

Despite this, the product manager is hesitant to change the API schema or add more documentation because they are concerned about the negative impact it might have on the existing integrations. They are worried that making changes to the schema or adding more documentation could cause confusion or disrupt the work of developers who are already using the API.

This hesitation to change the API is an example of status quo bias, as the product manager is inclined to maintain the current state of affairs rather than consider the potential benefits of making a change.

Outliers

Outliers are data points that are significantly different from the majority of data in a sample. When analyzing product metrics, outliers can have a significant impact on the results, as they can skew the mean and other statistical measures. There are several ways to handle outliers when analyzing product metrics.

• Ignore them: One option is simply to ignore the outliers and focus on the rest of the data. This may be appropriate if the outliers are not representative of the underlying population and are not expected to occur frequently.

• Exclude them: Another option is to exclude the outliers from the analysis. This can be done by setting a threshold for what is considered an outlier and excluding any data points that fall outside of that threshold. ...