Gain insights into managing risks in ML pipelines by exploring data issues, model biases, and security threats. Learn about data privacy, adversarial attacks, and alternative AI paradigms like causal AI and federated learning.

Disaster-Proofing Machine Learning Pipelines.png

The machine learning (ML) pipeline involves a complex relationship between the data, the model, and its implementation—each with its own risks that can adversely affect the utility and profitability of the solution. This course is a primer on what these risks are, where they come from, and how to mitigate them effectively.

In this course, you’ll start with a comprehensive look at the data side of the pipeline, including data privacy, data drift, and more. You’ll learn how to mitigate these in theory and practice. You’ll also discover problems related to ML models such as bias, security, and adversarial attacks. Finally, you’ll learn some of the alternative AI paradigms that exist in the world today—from causal AI to federated learning to generative AI.

A deep understanding of where problems can arise is a critical part of a data engineer or data scientist’s ML knowledge. From a career perspective, this course’s content can effectively address the real risks faced by developers while setting up ML pipelines.

Mitigating Disasters in ML Pipelines

# import libraries
import pandas as pd
import numpy as np

# set seed for reproducibility
np.random.seed(101)

# create fake data via a normal distribution
original_data = pd.DataFrame(data={
  'age': np.round(np.abs(np.random.normal(loc=50, scale=20, size=1000))),
  'credit_score': np.round(np.random.normal(loc=650, scale=50, size=1000)),
  'race': np.random.choice([0, 1], p=[0.80, 0.20], size=1000)
})

# get counts of minority class for oversampling
minority = original_data.loc[original_data['race'] == 1]
class_counts = original_data['race'].value_counts()
rows_to_add = class_counts[0] - class_counts[1]

print('Initial Distribution\n0: {}, 1: {}'.format(class_counts[0], class_counts[1]))

# oversample from minority class
sampled_rows = np.random.choice(range(minority.shape[0]), 
                                replace=True, 
                                size=rows_to_add)
altered_data = pd.concat([original_data, minority.iloc[sampled_rows]], axis=0)

class_counts = altered_data['race'].value_counts()
print('Oversampled Distribution\n0: {}, 1: {}'.format(class_counts[0], class_counts[1]))

print('\nOriginal Data Stats:\n{}'.format(original_data.describe()))
print('\nAltered Data Stats:\n{}'.format(altered_data.describe()))

\tau_{orig}=\frac{P[Z=1]}{P[Z=0]}=\frac{20\%}{80\%}=0.25 \rarr \tau_{alt}=\frac{50\%}{50\%}=1.00

Learn some mathematical approaches to bias mitigation in ML.

Introduction

Disasters in Data

Disasters in Models

Alternatives to Traditional ML

Conclusion

Assessment: Disasters in ML Pipelines

Theory of Data Bias Mitigation

Non-ML approaches