AI-powered learning
Save this course
Building a Machine Learning Pipeline from Scratch
Learn about ML pipeline development, delve into best practices, discover advanced Python concepts, and explore testing methodologies to elevate your software engineering skills and career prospects.
4.5
42 Lessons
2 Projects
14h
Join 2.9 million developers at
Join 2.9 million developers at
LEARNING OBJECTIVES
- An understanding of what constitutes as a machine learning training pipeline
- Hands-on experience building a machine learning pipeline in Python
- Familiarity with advanced Python concepts, such as abstract base classes and mixins
- An understanding of software engineering best practices, including code style, documentation, and logging
- A working knowledge of unit testing in Python
Learning Roadmap
2.
Getting Started
Getting Started
Look at how traditional engineering practices enhance ML pipeline stability and collaboration.
3.
Structuring the ML Pipeline
Structuring the ML Pipeline
6 Lessons
6 Lessons
Break apart the ML pipeline structure, directory organization, code style, and dependency management.
4.
Directed Acyclic Graphs (DAGs)
Directed Acyclic Graphs (DAGs)
3 Lessons
3 Lessons
Break down the steps to construct and sort DAGs for machine learning tasks.
5.
The ML Library
The ML Library
8 Lessons
8 Lessons
Unveil object-oriented implementation, configuration management, dataset and model handling, and report generation in ML pipelines.
6.
The Pipeline Core
The Pipeline Core
7 Lessons
7 Lessons
Follow the process of structuring machine learning pipelines, including argument parsing, logging, and tracking experiments.
7.
Extending the Pipeline
Extending the Pipeline
2 Lessons
2 Lessons
Build on extending machine learning pipelines to support new datasets and models effectively.
8.
Testing
Testing
4 Lessons
4 Lessons
Step through unit testing, Pytest for code coverage, and comprehensive system testing.
9.
Deployment
Deployment
2 Lessons
2 Lessons
Get started with packaging and deploying machine learning models for consistent predictions.
10.
Other Considerations
Other Considerations
4 Lessons
4 Lessons
Explore considerations for data quality monitoring, reproducibility, and leveraging off-the-shelf ML solutions.
Certificate of Completion
Showcase your accomplishment by sharing your certificate of completion.
Complete more lessons to unlock your certificate
Developed by MAANG Engineers
ABOUT THIS COURSE
Machine learning (ML) has matured into a mainstream development activity, and data scientists are expected to be able to write production-grade training pipelines. This course will provide you with a foundation in ML pipeline development guided by best practices in software engineering.
You’ll start by learning about code organization, style, and conceptual ideas, such as topological sorting of directed acyclic graphs. You’ll dive into the hands-on development of an ML pipeline. You’ll learn some advanced Python concepts and useful libraries. Finally, you’ll learn about testing methodologies and how to package your code into a distributable library.
By the end of this course, you’ll understand what makes a great ML pipeline work, better appreciate software engineering processes, improve your Python knowledge, and learn how to build ML training pipelines. This knowledge will make you more attractive to potential employers and improve your chances of thriving as a new data scientist.
ABOUT THE AUTHOR
Jayanth Chennamangalam
I am a full-stack data scientist based in Vancouver, British Columbia, Canada. Previously, I was an astronomer and a software engineer.
Trusted by 2.9 million developers working at companies
A
Anthony Walker
@_webarchitect_
E
Evan Dunbar
ML Engineer
S
Software Developer
Carlos Matias La Borde
S
Souvik Kundu
Front-end Developer
V
Vinay Krishnaiah
Software Developer
Built for 10x Developers
No Passive Learning
Learn by building with project-based lessons and in-browser code editor
Personalized Roadmaps
The platform adapts to your strengths & skills gaps as you go
Future-proof Your Career
Get hands-on with in-demand skills
AI Code Mentor
Write better code with AI feedback, smart debugging, and "Ask AI"
MAANG+ Interview Prep
AI Mock Interviews simulate every technical loop at top companies
Free Resources
