Learn Everything

Paths/MLOps Engineer/Model Serving / Scaling & Rollouts

Practice Taskintermediateubuntu:24.04Ask AI

Autoscale the Serving Tier

Model servers are slow to start and expensive to keep warm. Write the autoscaler that scales them on GPU utilisation instead of on request count.

Validator

⏱ 0:00

IDLE — checks run live as you type commands; submit when done

01Requirements

live checks
  • Directory ~/serving exists○
  • File ~/serving/hpa.yaml exists○
  • The config declares a HorizontalPodAutoscaler○
  • The config states the minimum and maximum replica counts○
  • The config scales on average utilisation○

02Walkthrough

1 of 4 revealed
  1. 01

    Create the serving directory

    Serving configuration lives next to the model server configuration.

    mkdir -p ~/serving
  2. 3 locked · each failed submit reveals one

03Stuck?

0 of 3 revealed

Take a hint without losing the lab — each one points at the thing to look at, not at the answer.

04AI Tutor

guides, never solves

Tell me what you're stuck on — I'll nudge, not solve. Which requirement are you looking at?

04Notebook

saved locally in this browser

learner@lab — ubuntu:24.04

↑/↓ recalls commands · Ctrl+Enter submits · type help for the command list

← Previous lab

Serve the Model on a Port

Model Serving

Next lab →

Roll Out a New Model Gradually

Model Serving