Delve into serverless computing, develop AWS Lambda Functions, and deploy an end-to-end machine learning pipeline. Gain insights into efficiently managing ML applications without infrastructure concerns.

Serverless 101 - Essential Patterns for Data Scientist.png

codes.tar.gz

AWS_ACCESS_KEY_ID

AWS_SECRET_ACCESS_KEY

AWS_REGION

AWS_OUTPUT

AWS_BUCKET_NAME

SERVERLESS_ORG

SERVERLESS_APP

SERVERLESS_SERVICE

DATA_PATH

aws_serverless

aws_serverless2

aws_serverless3

aws_serverless4

aws_test

Challenge2-B

Challenge2-A

Challenge2-B-S

Challenge2-A-S

Challenge3-S

Challenge3

Challenge2t

Challenge2t-S

Serverless computing makes the life of a developer easy by eliminating the pains of managing servers. You only need to write code—the serverless cloud services, such as AWS Lambda Functions, take care of infrastructure on their own. The Serverless framework provides easy to use commands and dashboards to make it even easier to use cloud services.

In this course, you’ll learn what serverless computing is and how the Serverless framework works. You’ll then get hands-on experience developing and deploying an AWS Lambda Function. Next, you’ll learn how to provide additional libraries using Layers in AWS Lambda. Having covered the basics, you’ll then deploy an end-to-end machine learning pipeline to the AWS cloud with a serverless approach.

Mastery over the serverless framework will give you the ability to focus on your machine learning problem and pipeline, instead of worrying about infrastructure management. This will help you get your machine learning applications to market faster and scale them seamlessly!

Mastering Serverless Computing for Data Science

## Solution

Here are the steps required to reproduce this solution:

1) Run the `train.py` script to train a model and generate the data files that you'll upload to Amazon S3 in the next step.
2) Create a bucket to store the train/ test data used for training the model. The Lambda function will be aware of this location. The value of the environment variable `DATA_PATH` is   `s3://BUCKET_NAME/`. Replace the `BUCKET_NAME` with a bucket name of your choice.
3) By default, a Lambda function has no permissions except for persisting its logs. For the Lambda function to list and read data from the previously created bucket, the default Lambda role permissions created automatically by Serverless have already been adjusted (see `iam` block in `provider` section). This gives the Lambda function additional minimal privileges for listing all buckets, and basic data operations on the created bucket.
4) Substitute `SERVERLESS_ORG`, `AWS_BUCKET_NAME` and `DATA_PATH`, and then deploy your own service.
5) Test the service in your Lambda's test window by providing the following payload where `partition` is equal to `test` or `train`. How different is the score?
```bash
{
  "partition": "test"
}
```



## Playground
Use the following widget to perform the abovementioned steps.

# Solution

Here are the steps required to reproduce this solution:

1) Run the `train.py` script to train a model and generate the data files that you'll upload to Amazon S3 in the next step.
2) Create a bucket to store the train/ test data used for training the model. The Lambda function will be aware of this location. The value of the environment variable `DATA_PATH` is   `s3://BUCKET_NAME/`. Replace the `BUCKET_NAME` with a bucket name of your choice.
3) By default, a Lambda function has no permissions except for persisting its logs. For the Lambda function to list and read data from the previously created bucket, the default Lambda role permissions created automatically by Serverless have already been adjusted (see `iam` block in `provider` section). This gives the Lambda function additional minimal privileges for listing all buckets, and basic data operations on the created bucket.
4) Substitute `SERVERLESS_ORG`, `AWS_BUCKET_NAME` and `DATA_PATH`, and then deploy your own service.
5) Test the service in your Lambda's test window by providing the following payload where `partition` is equal to `test` or `train`. How different is the score?
```bash
{
  "partition": "test"
}
```



# Playground
Use the following widget to perform the abovementioned steps.

Let's take a look at the solution for this exercise.

Solution: Train and Evaluate ML Model

Basics of Serverless Computing

ML Model Dockers in AWS Lambda

Serverless Architecture for Data Science

Conclusion

Appendix

Solution: Train and Evaluate ML Model

Solution

Playground