Module · Data Engineer
Governance & Lineage
Who owns the data, where it came from, and what must never leave the building.
TOPIC 01
Catalog & Lineage
The entry that says what a dataset is, and the graph that says what depends on it.
What you'll practice
- Write a Catalog Entry — guided · beginner
- Map the Lineage — practice · intermediate
A command you'll meet
cat ~/governance/catalog/users.yamlA dataset nobody can find is a dataset nobody trusts. Open the task below for the full walkthrough.
Hands-On Tasks in This Topic (2)
Write a Catalog Entry
The catalog entry is where a dataset earns its name - who owns it, what it holds and how sensitive it is. Write one.
Map the Lineage
Lineage is the answer to what breaks when this table changes. Write the graph of upstream sources and downstream consumers.
TOPIC 02
Privacy & Retention
Know what is personal, mask it in the right places, and delete it when you must.
What you'll practice
- Register the Personal Data — practice · intermediate
- Write the Retention Policy — challenge · advanced
A command you'll meet
cat ~/governance/pii.yamlPersonal data is a liability with a column name. Open the task below for the full walkthrough.
Hands-On Tasks in This Topic (2)
Register the Personal Data
You cannot mask what you have not listed. Write the registry of the personal fields in the warehouse and what happens to each one.
Write the Retention Policy
Keeping data forever is not neutral - it is a growing liability. Write the retention windows and the jobs that enforce them.