06.07 · Short
Catalogs, Lineage, and Access Control
Configure a conceptual data catalog entry with ownership, schema metadata, lineage, tags, and access policies.
A useful data catalog entry records ownership, structure, origin, meaning, sensitivity and permitted use for a warehouse or lakehouse asset. Lineage shows how data moves through sources, jobs and consumers, while access policies use roles, groups and tags to keep discovery broad but actual data use controlled.
What this lesson answers
- what belongs in a data catalog entry
- how lineage helps debug broken data pipelines
- RBAC versus ABAC for data access policies
Notes
A data catalog is the operational inventory for a warehouse or lakehouse: it tells engineers and analysts what data exists, what it means, who owns it, how it is structured, where it came from, and how it may be used. A good catalog entry is more than a table name and description. It should include a clear business owner and technical owner, the physical location or system of record, schema metadata such as column names, data types, descriptions, nullability, and key fields, plus tags that make the asset searchable and governable.
Common questions
- What metadata should a data catalog store for a table?
- A practical catalog entry should name the business and technical owners, describe the dataset’s purpose, record its source or physical location, and capture schema details such as columns, types, keys and nullability. It should also include governance tags for domain, sensitivity, retention and quality so people and policies can find and classify it.
- Why does lineage matter in a warehouse or lakehouse?
- Lineage makes dependencies visible. When an upstream source changes, engineers can see which transformations, dashboards, exports or feature tables are at risk. When a number looks wrong, they can trace it back through ingestion and processing steps instead of relying on memory or asking around for hidden context.
- How do catalog tags affect access control?
- Tags let access rules follow the data rather than only specific tables or storage paths. A sensitivity tag can trigger stricter read rules, masking, approval flows or limited sharing. Combined with ownership and schema metadata, tags support policies based on roles, groups, domains and regulated fields.