Data Science in Production: Building Scalable Model Pipelines/

...

/

- Cloud Cron

- Cloud Cron

Example of Cron in GKE.

We'll cover the following...

Introduction

Both AWS and GCP provide managed cron options. With AWS it’s possible to schedule services to run on ECS using cron expressions, and the GCP Kubernetes Engine provides scheduling support for containers.

Cloud data

Crons in GKE

In this lesson, we’ll explore the GKE option because it is simpler to set up. The first step is to push our model pipeline image to Container Registry using the following commands:

Press + to interact
cat dsdemo.json | sudo docker login -u _json_key --password-stdin https://us.gcr.io
sudo docker tag sklearn_pipeline us.gcr.io/[project_id]/sklearn_pipeline
sudo docker push us.gcr.io/[project_id]/sklearn_pipeline

We will be performing these steps in the code widget at the end of the lesson.

Creating a cluster

Next, we’ll set up Kubernetes to schedule the image to run on a managed cluster. Unlike in ...