Paths/Data Engineer/Streaming & Event Pipelines / Delivery Semantics
Handle Late and Out-of-Order Events
Network delays mean events arrive late. Decide how long you wait, and what happens to the ones that arrive after the deadline.
Validator
IDLE — checks run live as you type commands; submit when done
01Requirements
live checks- File ~/streaming/watermark.yaml exists○
- The config states the watermark delay○
- The config states how long the engine waits before closing a window○
- The config states what happens to events that arrive later○
02Walkthrough
1 of 3 revealed- 01
Write the watermark configuration
watermark_delay_ms is how long the engine waits for stragglers before it closes a window. Too short and you lose events, too long and you add latency.
echo 'watermark:\n watermark_delay_ms: 30000\n allowed_lateness_ms: 60000\n on_late_event: update_aggregate' > ~/streaming/watermark.yaml - 2 locked · each failed submit reveals one
03Stuck?
0 of 3 revealedTake a hint without losing the lab — each one points at the thing to look at, not at the answer.
04AI Tutor
guides, never solvesTell me what you're stuck on — I'll nudge, not solve. Which requirement are you looking at?
04Notebook
saved locally in this browserlearner@lab — ubuntu:24.04
Session is disposable
↑/↓ recalls commands · Ctrl+Enter submits · type help for the command list