Paths/AI Engineer/LLM Evaluation / Metrics & Release Gates
Gate Releases on Eval Results
A gate is a rule that fails the build. Write the gate configuration - the accuracy floor, the regression limit and what happens when either is broken.
Validator
⏱ 0:00
IDLE — checks run live as you type commands; submit when done
01Requirements
live checks- File ~/evals/gate.yaml exists○
- The gate sets a minimum accuracy○
- The gate sets a maximum allowed regression○
- The gate fails the build when a limit is broken○
02Walkthrough
1 of 3 revealed- 01
Write the gate configuration
min_accuracy is the floor the run must clear, max_regression is how much worse the new version may be than the baseline.
echo '{"baseline": "v12", "min_accuracy": 0.9, "max_regression": 0.02, "on_failure": "block_release"}' > ~/evals/gate.yaml - 2 locked · each failed submit reveals one
03Stuck?
0 of 3 revealedTake a hint without losing the lab — each one points at the thing to look at, not at the answer.
04AI Tutor
guides, never solvesTell me what you're stuck on — I'll nudge, not solve. Which requirement are you looking at?
04Notebook
saved locally in this browserlearner@lab — ubuntu:24.04
Session is disposable
↑/↓ recalls commands · Ctrl+Enter submits · type help for the command list