Explore serverless computing with AWS Lambda, deploy ML pipelines, and manage applications efficiently without infrastructure worries.

Serverless 101 - Essential Patterns for Data Scientist.png

codes.tar.gz

AWS_ACCESS_KEY_ID

AWS_SECRET_ACCESS_KEY

AWS_REGION

AWS_OUTPUT

AWS_BUCKET_NAME

SERVERLESS_ORG

SERVERLESS_APP

SERVERLESS_SERVICE

DATA_PATH

aws_serverless

aws_serverless2

aws_serverless3

aws_serverless4

aws_test

Challenge2-B

Challenge2-A

Challenge2-B-S

Challenge2-A-S

Challenge3-S

Challenge3

Challenge2t

Challenge2t-S

Serverless computing makes the life of a developer easy by eliminating the pains of managing servers. You only need to write code—the serverless cloud services, such as AWS Lambda Functions, take care of infrastructure on their own. The Serverless framework provides easy to use commands and dashboards to make it even easier to use cloud services.

In this course, you’ll learn what serverless computing is and how the Serverless framework works. You’ll then get hands-on experience developing and deploying an AWS Lambda Function. Next, you’ll learn how to provide additional libraries using Layers in AWS Lambda. Having covered the basics, you’ll then deploy an end-to-end machine learning pipeline to the AWS cloud with a serverless approach.

Mastery over the serverless framework will give you the ability to focus on your machine learning problem and pipeline, instead of worrying about infrastructure management. This will help you get your machine learning applications to market faster and scale them seamlessly!

Mastering Serverless Computing for Data Science


Now that we've reached the end of this chapter, its time for you to write some ML model code. We recommend following a more rigorous data science workflow this time, e.g., `scikit-learn` pipelines.

# Problem statement
Write a script that would train and persist a model locally, as well as the train/test data used for it. Then upload your artifacts to Amazon S3 and write AWS Lambda code that will evaluate your model performance on train/test datasets. We have already split the dataset into train/test files. Please see a more detailed list of steps below:

* Choose a real-life dataset (the wine quality dataset) and follow a typical data science routine:
  * Write data preprocessing functions.
  * Optimize the parameters, get the best model, and get train dataset evaluation metrics for your dataset. For that, we recommend using `scikit-learn` pipelines to combine data preprocessing and model training. The result should be such that your `train.py` script produces a `model.joblib` file that you will evaluate in your Lambda function. You can then upload the train/test files on an Amazon S3 bucket, from which they will be read by your function.

* Write your own `handler.py` Lambda function. It should either be able to load your newly trained ML model from Amazon S3 or have the model already available in the container and calculate model performance metrics based on the train/test datasets imported from S3. You will need to consider how to pass information about the file's location on Amazon S3 to your function. 

## Playground

Test yourself by performing the abovementioned tasks in the following widget:



Test the knowledge you've gained so far by applying it to this hands-on exercise.

Exercise: Train and Evaluate ML Model

Basics of Serverless Computing

ML Model Dockers in AWS Lambda

Serverless Architecture for Data Science

Conclusion

Appendix

Scalable Machine Learning Model for Accurate Predictions on AWS

Exercise: Train and Evaluate ML Model

Problem statement