Data vs. Model Bias

Introduction to bias sources

Before diving into bias mitigation methods, it’s essential to analyze potential sources of bias. Understanding these sources will enable us to make informed decisions about selecting appropriate metrics and fixing methods. We can broadly categorize the primary sources of bias in machine learning into two categories:

Data bias: When the data fed to the model is biased, the model reproduces these biases.

Model bias: When the model or modeling technique itself introduces bias.

Data bias

Let’s begin by examining data bias:

• Sampling bias: This occurs when data is collected in a non-representative manner, and the sample distribution does not represent the population distribution. For example, an internet survey may only capture responses from internet-proficient people.

• Labeling bias: This occurs when humans label examples. For instance, a hate speech classifier dataset might suffer from the subjective judgment of annotators. Datasets covering controversial topics are particularly vulnerable to this issue.

• Measurement bias: Even if data is collected using machines, measurement errors can still introduce unfairness. For example, two different devices may be used to measure two populations, leading to inconsistencies.

• Confirmation bias: This arises when data is chosen in a way that confirms pre-existing expectations. If data is collected to prove a specific hypothesis, this bias will likely affect it.

• Temporal bias: Data might become outdated and no longer reflect current reality. It is particularly dangerous when data for different subgroups are collected at other time windows.

• Reporting bias: This occurs if some data points are more likely to be reported than others. For example, imagine an adverse events reporting procedure that isn’t followed consistently across different hospitals. This can create a false impression of one hospital’s performance relative to another.

Addressing bias in the dataset can significantly improve the overall fairness of the final solution. Awareness of these issues can save a lot of trouble during development if the data has not yet been collected. Moreover, fixing biases at this stage will benefit any model used later in the modeling phase. However, often, the data has already been collected, and not much can be done about it.

Model bias

Even if the dataset is completely unbiased (which is unlikely), the model can still introduce biases on its own. The severity of this issue will depend on the specific dataset and model type. Some algorithms and modeling techniques are more prone to exploiting certain patterns, resulting in an unfair model. Let’s examine some examples of bias introduced by the model:

• Inductive bias: This refers to the set of assumptions about the relationship between features and labels that the model has to make to learn. For example, linear regression assumes that the output is a linear function of the input. However, such assumptions might not hold true in reality, leading to biased reasoning.

  • Relational/non-relational bias: A specific subtype of inductive bias. It is related to predictions of specific relations (or lack of such predictions). For example, a model for forecasting workplace promotions can be more likely to favor males over females if such an imbalance is present in the dataset. For instance, linear regression can assign a high coefficient to feature gender. 

  • Learning bias: As training is performed iteratively for most models, they result in local optima instead of global. Specific training procedures tend to favor a particular minimal over another, resulting in a biased model.