Designing Machine/Deep Learning Models Using Azure CLI/

...

/

Creating an E2E ML Pipeline

Creating an E2E ML Pipeline

Create a machine learning pipeline in Azure.

We'll cover the following...

What is a pipeline job?

Often, building ML models requires stitching multiple jobs together. For example, we might want to run two jobs in sequential order. We can stitch multiple steps into a single Azure pipeline in such a case. We will define a new job type pipeline and the sequential steps in the YAML file.

We can see a pipeline command example below. We have two jobs: hello_job and world_job, which run in sequence as a single pipeline. Let’s run this job.

$schema: https://azuremlschemas.azureedge.net/latest/pipelineJob.schema.json
type: pipeline
jobs:
  hello_job:
    command: echo "hello"
    environment:
      image: library/python:latest
    compute: azureml:ComputeEdu
  world_job:
    command: echo "world"
    environment:
      image: library/python:latest
    compute: azureml:ComputeEdu
Running a simple hello world pipeline job
...