MLOps
Model Tracking & MLflow
Experiment tracking, parameter logging, model registries, and artifact lineage.
Machine learning development is empirical and iterative. MLflow provides an open-source platform to track training hyperparameters, metrics curves, model weights, and data lineage across teams, streamlining the progression from notebook experiment to registered production artifact.
Virtual Lab Ready
Automated testing sandbox running in your browser.
Launch First Lab ⚡No local installation required
Hands-on Practice
Interactive Terminal Labs
Real environment tasks with automated validation. Write real commands, verify state, and debug errors.
Configure an MLflow Tracking Server Manifest
Topic: Experiment Tracking with MLflow
Set up an MLflow tracking server configuration with backend SQLite store and artifact S3 bucket path.
Key Checks:
- Directory ~/mlflow exists
- File ~/mlflow/tracking-server.sh exists
Register the Champion Model
Topic: The Model Registry
Lay out a registry at ~/registry with a versioned home for the finance-net model, then mark v3 as the Production champion. This is the structure MLflow's registry automates.
Key Checks:
- ~/registry/models/finance-net/v3 exists
- stage.txt marks v3 as stage=Production
Tag Every Model Version
Topic: Model Lineage & Promotion
Tags are the lineage of a model - the code, the data and the owner behind a set of weights. Define the tag set every version must carry.
Key Checks:
- Directory ~/mlflow exists
- File ~/mlflow/tags.yaml exists
Promote Models with Aliases
Topic: Model Lineage & Promotion
Stages move models through the registry, aliases point at them. Write the promotion rules that decide which version serves traffic.
Key Checks:
- File ~/mlflow/aliases.yaml exists
- The file maps the champion and challenger aliases to versions
System Design
MLflow Tracking Server & Model Registry
Understand how this tool operates internally before running production workloads.
Internal Mechanics
Data scientists log parameters and metrics via the Python SDK. The tracking server persists metadata into a SQL database (PostgreSQL/SQLite) and stores heavy weight binaries in object storage (S3/GCS).
Mental Model Rule of Thumb
Never treat Model Tracking & MLflow as a black box. Understanding its boundary with the kernel, filesystem, and network stack prevents 90% of production outages.
Core Architectural Subsystems
Tracking Server API
Central REST server receiving run metrics, parameters, and tags.
Backend Store (PostgreSQL)
Stores tabular metadata: run IDs, experiment names, metrics histories.
Artifact Store (S3/MinIO)
Stores serialized model files (.pt, .onnx), graphs, and requirements.txt.
Model Registry
Provides stage transitions (Staging, Production, Archived) and semantic versioning.
Developer Reference
Command Reference & Cheatsheet
Curated commands, flags, and one-liners tested for everyday operational productivity.
mlflow server --backend-store-uri postgresql://... --default-artifact-root s3://my-bucketLaunch production tracking server with Postgres and S3
mlflow ui --port 5000Launch local web UI to compare loss curves and parameters
Fundamental Primitives
Key Terminology & Primitives
Core abstractions you must master to communicate and build with confidence.
Run vs Experiment
An experiment groups related efforts (e.g. sentiment-classifier); a run is a single training job execution.
Autologging
Automatic extraction of metrics, parameters, and models from frameworks like PyTorch, Scikit-learn, and XGBoost.
Model Signature
Input and output data type schema attached to a model so clients validate payload formats before inference.
Reproducibility
Recording exact Git commit SHA, dataset version, and environment dependencies for every run.
Production War-Stories
Common Pitfalls & Debugging Playbook
The exact errors that take down production systems and how to diagnose and resolve them.
Artifact upload timeout on large weights
Symptom
Failed to upload model weights exceeding multiple gigabytes.
Resolution
Configure S3 multipart uploads and ensure tracking server has adequate network bandwidth and IAM permissions.
Reliability Standards
Production Readiness Checklist
Review this checklist before deploying workloads to staging or production clusters.
- ✓Log input dataset hashes or DVC commit hashes alongside model metrics.
- ✓Enforce mandatory model signatures on all registered candidates.
- ✓Automate deployment promotion via webhooks triggered on Model Registry stage changes.
- ✓Back up tracking server metadata database regularly.
Full Curriculum Track
Master Model Tracking & MLflow as part of the MLOps Engineer path
Tools do not exist in isolation. Learn how Model Tracking & MLflow connects with other systems, CI/CD pipelines, and infrastructure patterns.