Answer-first summary for fast verification
Answer: Develop a custom metadata catalog that integrates with the lakehouse, tracking schema changes and lineage information.
In scenarios with rapidly evolving data sources and formats, a scalable and efficient metadata management strategy is essential. Developing a custom metadata catalog that integrates with the lakehouse (Option B) provides a robust solution by tracking schema changes and lineage information. This approach is superior to relying solely on the lakehouse's built-in capabilities (Option C) or manual updates (Option D), as it ensures metadata is easily accessible and understandable. While external schema registry services (Option A) offer some benefits, a custom catalog integrated with the lakehouse offers a more seamless and comprehensive solution.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
In a lakehouse architecture with rapidly evolving data sources and formats, what strategy ensures scalable and efficient metadata management?
A
Use external schema registry services to manage metadata independently from the lakehouse storage layer.
B
Develop a custom metadata catalog that integrates with the lakehouse, tracking schema changes and lineage information.
C
Rely solely on the lakehouse‘s built-in schema inference and evolution capabilities without additional metadata management practices.
D
Manually update metadata in a centralized documentation repository, relying on data engineers to consult this repository during data ingestion and processing.