Module · MLOps Engineer
ML Pipelines
Data and training pipelines that rerun reliably.
TOPIC 01
Your First Pipeline
Declare a training DAG as files, then package it as a container image.
What you'll practice
- Declare the Pipeline DAG — guided · beginner
- Containerize the Pipeline — practice · intermediate
A command you'll meet
docker build -t pipeline:v1 .A pipeline step that only runs on your laptop is a rumor. Packaging it as an image makes it runnable anywhere — including the next topic's orchestrator. Open any task below for the full step-by-step walkthrough.
Hands-On Tasks in This Topic (2)
Declare the Pipeline DAG
Lay out ~/pipeline with one file per step and a pipeline.yaml that names the execution order — the shape every orchestrator (Airflow, Dagster, Kubeflow) recognises instantly.
Containerize the Pipeline
Turn the pipeline into a runnable image. Building it yourself is how you learn why images — not laptops — are the unit of execution in MLOps.
TOPIC 02
The Training Pipeline
The stage that fits the model, and the stage that publishes what it produced.
What you'll practice
- Define the Training Stage — practice · intermediate
- Publish the Model Artifact — challenge · advanced
A command you'll meet
cat ~/pipeline/stages/train.yamlA stage without declared inputs and outputs cannot be retried safely. Open the task below for the full walkthrough.
Hands-On Tasks in This Topic (2)
Define the Training Stage
The training stage is where the DAG stops being a plan and starts doing work. Write its definition with the image, the command and the artifacts it consumes and produces.
Publish the Model Artifact
A trained model is not a deliverable until it is stored, signed and checked. Write the publish stage that does all three.