Paths/MLOps Engineer/Monitoring / Metrics & Drift
Log the Quality Metrics
Create ~/monitoring with a metrics file and an explicit alerting threshold — because a number is only meaningful once you have written down what it is allowed to drop to.
Validator
IDLE — checks run live as you type commands; submit when done
01Requirements
live checks- metrics.txt records accuracy=0.87○
- threshold.txt records threshold=0.85○
02Walkthrough
6 steps- 01
Create the monitoring directory
Monitoring artifacts deserve their own home, separate from training code.
mkdir ~/monitoring - 02
Move into it
Everything below is created relative to ~/monitoring.
cd ~/monitoring - 03
Log the accuracy
One honest number from the latest evaluation run. Key=value format stays readable in any dashboard or diff.
echo accuracy=0.87 > metrics.txt - 04
Write the alerting threshold
The threshold is the contract — accuracy is allowed to fall to 0.85, no further, without someone getting paged.
echo threshold=0.85 > threshold.txt - 05
Review both files
cat prints a file. Check both — the metric and the contract it must honour.
cat metrics.txt threshold.txt - 06
Submit for validation
The validator checks both files exist with the exact metric and threshold values.
03Stuck?
0 of 3 revealedTake a hint without losing the lab — each one points at the thing to look at, not at the answer.
04AI Tutor
guides, never solvesTell me what you're stuck on — I'll nudge, not solve. Which requirement are you looking at?
04Notebook
saved locally in this browserlearner@lab — ubuntu:24.04
Session is disposable
↑/↓ recalls commands · Ctrl+Enter submits · type help for the command list