Paths/Data Engineer/Streaming & Event Pipelines / Event & Topic Design
Version an Event Schema
Every event needs a schema, a version and a timestamp - consumers build their state from it. Write the contract for the order placed event.
Validator
IDLE — checks run live as you type commands; submit when done
01Requirements
live checks- Directory ~/streaming/schemas exists○
- File ~/streaming/schemas/order_placed.json exists○
- The schema names the event and its version○
- The schema includes the identifier, the timestamp and the payload○
02Walkthrough
4 steps- 01
Create the schemas directory
One file per event - the directory is the registry.
mkdir -p ~/streaming/schemas - 02
Write the event contract
The name and the version travel with every message. Consumers branch on the version, not on the shape they happen to see.
echo '{"name": "order_placed", "version": 2, "key": "order_id", "fields": {"order_id": "string", "customer_id": "string", "amount": "number", "occurred_at": "timestamp"}}' > ~/streaming/schemas/order_placed.json - 03
Verify the contract
cat prints the schema. The key field decides the partitioning - choose it now.
cat ~/streaming/schemas/order_placed.json - 04
Submit for validation
The validator checks the event name, the version and the required fields.
03Stuck?
0 of 3 revealedTake a hint without losing the lab — each one points at the thing to look at, not at the answer.
04AI Tutor
guides, never solvesTell me what you're stuck on — I'll nudge, not solve. Which requirement are you looking at?
04Notebook
saved locally in this browserlearner@lab — ubuntu:24.04
Session is disposable
↑/↓ recalls commands · Ctrl+Enter submits · type help for the command list