Module · MLOps Engineer
Monitoring
Drift, latency and the health of ML systems.
TOPIC 01
Metrics & Drift
Log model quality honestly, then detect when the world moves under it.
What you'll practice
- Log the Quality Metrics — guided · beginner
- Call the Drift Verdict — challenge · intermediate
A command you'll meet
echo accuracy=0.87 > metrics.txtA model without a logged metric is a claim without evidence. A model whose metric is never re-checked is a claim that quietly expires. These two tasks are the smallest honest monitoring loop. Open any task below for the full walkthrough.
Hands-On Tasks in This Topic (2)
Log the Quality Metrics
Create ~/monitoring with a metrics file and an explicit alerting threshold — because a number is only meaningful once you have written down what it is allowed to drop to.
Call the Drift Verdict
The feature distribution moved — baseline mean 0.42, current mean 0.61. Record both sides in ~/drift-lab and write the verdict file the on-call engineer will actually read.
TOPIC 02
Alerting & Retraining
The thresholds that page someone, and the drift that starts a new training run.
What you'll practice
- Set the Alert Thresholds — practice · intermediate
- Trigger Retraining on Drift — challenge · advanced
A command you'll meet
cat ~/monitoring/alerts.yamlAn alert without a threshold is a feeling with a notification. Open the task below for the full walkthrough.
Hands-On Tasks in This Topic (2)
Set the Alert Thresholds
Two numbers decide whether a model is healthy - how accurate it is and how fast it answers. Write the alert rules that watch both.
Trigger Retraining on Drift
When the data moves, the model has to move with it. Write the trigger that opens a retraining run once the drift score crosses a threshold.