Home>Courses>Data Science in Production: Building Scalable Model Pipelines

Data Science in Production: Building Scalable Model Pipelines

Gain insights into building scalable data and model pipelines, explore different cloud environments, delve into streaming workflows, and discover essential tools for creating real-time data products.

Advanced

110 Lessons

Certificate of Completion

Gain insights into building scalable data and model pipelines, explore different cloud environments, delve into streaming workflows, and discover essential tools for creating real-time data products.

AI-POWERED

Explanations

AI-POWERED

Explanations

This course includes

4 Cloud Labs

53 Playgrounds

16 Quizzes

Course Overview

Course Content

Apply Your Skills

Recommendations

Course Overview

The goal of this course is to provide you with a set of tools that can be used to build predictive model services for product teams. In this course, you’ll start by covering the different cloud environments and tools for building scalable data and model pipelines. You’ll then learn the different data sets and types of models that will be used heavily in everyday production. Throughout the course, you’ll have plenty of exercises and challenges to get you comfortable working with the diverse toolset. Lastly...Show More

Course Content

Introduction to Building Scalable Model Pipelines

18 Lessons

Get familiar with building scalable data pipelines, using Python, clouds, coding environments.

Course Overview

Applied Data Science

Python for Scalable Compute

Cloud Environments

Coding Environments

Quiz - Data Science Preliminary Concepts

🛑 Important Note!

Introduction to Datasets

- BigQuery to Pandas

- Kaggle to Pandas

Quiz - Importing Datasets

Prototype Models

- Linear Regression

- Logistic Regression

- Keras Regression

Automated Feature Engineering

Quiz - Prototype Modeling

Conclusion : Data science models, tools and environments

Models as Web Endpoints

12 Lessons

Solve challenges with deploying machine learning models as scalable web endpoints.

Introduction to Models as Web Endpoints

Quiz - Web Services and Persistent Models

Deploying a Web Endpoint

- Gunicorn

- Heroku

Interactive Web Services with Dash

Quiz - Web Deployments

Conclusion : Models as Web Endpoints

Models as Serverless Functions

17 Lessons

Go hands-on with deploying machine learning models using serverless functions on AWS and GCP.

Introduction to Models as Serverless Functions

Managed Services

Cloud Functions (GCP)

- Echo Service

- Cloud Storage (GCS)

Quiz - Managed Services

Lambda Functions (AWS)

- Echo Function

- Simple Storage Service (S3)

- Model Function

- API Gateway

Quiz - Lambda Functions

Conclusion

Create an Echo Function in Lambda

Cloud Lab

Working with S3 in Lambda

Cloud Lab

Working with API in Lambda

Cloud Lab

Containers for Reproducible Models

9 Lessons

Grasp the fundamentals of using containers to ensure reproducible, scalable model deployments.

Introduction to Containers as Reproducible Models

Docker

Orchestration

- AWS Container Registry (ECR)

- AWS Container Service (ECS)

- Load Balancing

Kubernetes on GCP

Quiz - Containers

Conclusion: Containers as Reproducible Models

Working with AWS Container Registry

Cloud Lab

Workflow Tools for Model Pipelines

10 Lessons

Take a closer look at managing and automating model pipelines with workflow tools like Airflow.

Introduction to Workflow Tools for Model Pipelines

Sklearn Workflow

Cron

- Cloud Cron

Quiz - Workflows and Scheduling

Workflow Tools

- Apache Airflow

- Managed Airflow

Quiz - Workflow Tools

Conclusion : Workflow Tools for Model Pipelines

10.

PySpark for Batch Pipelines

25 Lessons

Follow the process of building scalable model pipelines using PySpark in cloud environments.

Introduction to PySpark for Batch Pipelines

Spark Environments

- Spark Clusters

- Databricks Community Edition

Staging Data

Quiz - Spark Environments

A PySpark Primer

- Persisting Dataframes

- Converting Dataframes

Distributed Deep Learning

Distributed Feature Engineering

Quiz - Machine Learning in PySpark

Quiz - GCP Model Pipeline

Conclusion : Working tools for model pipelines

11.

Cloud Dataflow for Batch Modeling

8 Lessons

Master the creation and execution of scalable Cloud Dataflow pipelines for batch modeling.

Introduction to Cloud Dataflow and Batch Modeling

Quiz - Dataflow and Batch Modeling

Conclusion : Cloud Dataflow for Batch Modeling

12.

Streaming Model Workflows

10 Lessons

Learn how to use streaming platforms for scalable real-time machine learning pipelines.

Introduction to Streaming Model Workflows

Spark Streaming

- Apache Kafka

- Sklearn Streaming 1

- Sklearn Streaming 2

Dataflow Streaming

- PubSub

- Natality Streaming

Quiz - Streaming Model Workflows

Conclusion : Streaming model workflows

13.

Course Conclusion

1 Lessons

Look at the evolving journey in data science, emphasizing continuous learning and practice.

Thank You

Course Author

Ben Weber

Cloud Lab

Managing Real-Time Data with AWS AppSync Subscriptions

intermediate

1 hour 30 min

Cloud Lab

Managing Real-Time Data with AWS AppSync Subscriptions

Learn to manage real-time GraphQL APIs with AWS AppSync, integrating them into a Flutter chat application.

intermediate

1 hour 30 mins

Cloud Lab

Migrate an Internet-Scale Online Transactional System to DynamoDB

beginner

2 hour 30 min

Cloud Lab

Migrate an Internet-Scale Online Transactional System to DynamoDB

Learn to migrate a MySQL database to DynamoDB using AWS Database Migration Service in this Cloud Lab.

beginner

2 hours 30 mins

Trusted by 2.5 million developers working at companies

"These are high-quality courses. Trust me. I own around 10 and the price is worth it for the content quality. EducativeInc came at the right time in my career. I'm understanding topics better than with any book or online video tutorial I've done. Truly made for developers. Thanks"

Anthony Walker

@_webarchitect_

"Just finished my first full #ML course: Machine learning for Software Engineers from Educative, Inc. ... Highly recommend!"

Evan Dunbar

ML Engineer

"You guys are the gold standard of crash-courses... Narrow enough that it doesn't need years of study or a full blown book to get the gist, but broad enough that an afternoon of Googling doesn't cut it."

Carlos Matias La Borde

Software Developer

"I spend my days and nights on Educative. It is indispensable. It is such a unique and reader-friendly site"

Souvik Kundu

Front-end Developer

"Your courses are simply awesome, the depth they go into and the breadth of coverage is so good that I don't have to refer to 10 different websites looking for interview topics and content."

Vinay Krishnaiah

Software Developer

"I've tried probably 5-7 different sites and Educative is easily the best. It perfectly blends explanation with interactivity"

Eric Downs

Musician/Entrepeneur

Hands-on Learning Powered by AI

See how Educative uses AI to make your learning more immersive than ever before.

Instant Code Feedback

Evaluate and debug your code with the click of a button. Get real-time feedback on test cases, including time and space complexity of your solutions.

AI-Powered Mock Interviews

Put your skills to the test in a simulated interview setting. Receive personalized feedback based on your performance. Available in Premium & Premium Plus plans.

Adaptive Learning

At various checkpoints throughout Educative courses, you will be prompted to take a quick assessment. Receive a condensed curriculum tailored to your strengths and skill gaps.

Explain with AI

Select any text within any Educative course, and get an instant explanation — without ever leaving your browser.

AI Code Mentor

AI Code Mentor helps you quickly identify errors in your code, learn from your mistakes, and nudge you in the right direction — just like a 1:1 tutor!

Recommended before starting this course

Course

Web Security and Access Management: JWT, OAuth2 & OpenId Connect

Gain insights into JWT, OAuth2, and OpenID Connect. Learn about HTTPS, encryption, and handshake techniques, and explore foundational concepts in web API security for enhanced application protection.

intermediate

5 hours

Course

Web Application Security for the Everyday Software Engineer

Gain insights into enforcing web app security best practices, such as HTTPS, defending against XSS and clickjacking, managing HTTP cookies, and warding off DDoS attacks.

intermediate

4 hours

Course

Building Full-Stack Web Applications with AdonisJs

Gain insights into AdonisJs fundamentals. Delve into routes, controllers, middleware, and hooks. Discover practical examples, illustrations, and quizzes to confidently build your full-stack applications.

intermediate

Course

Web Security and Access Management: JWT, OAuth2 & OpenId Connect

intermediate

5 hour

Course

Web Application Security for the Everyday Software Engineer

intermediate

4 hour

Course

Building Full-Stack Web Applications with AdonisJs

intermediate

Related Courses and Skill Paths

Course

The Hacker's Guide to Scaling Python

Gain insights into Python's scalability, concurrency, and distribution. Learn about CPU scaling, event loops, queue-based distribution, and building REST APIs to optimize and deploy high-performance applications.

advanced

8 hours 30 mins

Course

Applied Machine Learning: Deep Learning for Industry

Gain insights into industry-level machine learning by applying advanced TensorFlow and Python techniques. Explore efficient methods for creating scalable models, from data pipelines to deployment and inference.

advanced

3 hours

Course

Make Your First GAN Using PyTorch

Gain insights into Generative Adversarial Networks with PyTorch. Delve into neural networks, adversarial training, and building GANs for image generation, including conditional GANs for class-specific data.

advanced

13 hours

Course

Feature Engineering for Machine Learning

advanced

30 mins

Course

The Hacker's Guide to Scaling Python

advanced

8 hour 30 min

Course

Applied Machine Learning: Deep Learning for Industry