Paths/AI Engineer/LLM Evaluation / Metrics & Release Gates
Report the Metrics of a Run
A run is only useful if it produces comparable numbers. Write the report with the accuracy, the latency and the cost per request.
Validator
⏱ 0:00
IDLE — checks run live as you type commands; submit when done
01Requirements
live checks- File ~/evals/report.json exists○
- The report states the accuracy of the run○
- The report states the p95 latency○
- The report states the cost per request○
02Walkthrough
1 of 3 revealed- 01
Write the run report
accuracy says whether the answers were right, p95_latency_ms says whether the experience was fast enough, cost_per_request_usd says whether it was worth it.
echo '{"model": "gpt-4o-mini", "cases": 50, "accuracy": 0.92, "p95_latency_ms": 1450, "cost_per_request_usd": 0.0007}' > ~/evals/report.json - 2 locked · each failed submit reveals one
03Stuck?
0 of 3 revealedTake a hint without losing the lab — each one points at the thing to look at, not at the answer.
04AI Tutor
guides, never solvesTell me what you're stuck on — I'll nudge, not solve. Which requirement are you looking at?
04Notebook
saved locally in this browserlearner@lab — ubuntu:24.04
Session is disposable
↑/↓ recalls commands · Ctrl+Enter submits · type help for the command list