Databricks Certified Data Engineer - Professional
Get started today
Ultimate access to all questions.
Ultimate access to all questions.
In the context of designing a multiplex bronze table for productionalizing streaming workloads on Azure Databricks, consider the following scenario: Your organization is ingesting streaming data from multiple sources, each with different formats and schemas. The goal is to ensure efficient data ingestion, processing, and maintain data consistency and integrity while adhering to cost constraints and scalability requirements. Which of the following approaches is the BEST to avoid common pitfalls in this scenario? (Choose one option)
Explanation:
The BEST approach to avoid common pitfalls when productionalizing streaming workloads is to design a single bronze table with a unified schema that can accommodate all streaming data sources and formats. This approach leverages Delta Lake's capabilities for efficient data ingestion, processing, and ensures data consistency and integrity. It simplifies the architecture by reducing the complexity of managing multiple tables and orchestration layers, while also being cost-effective and scalable. Key steps include identifying data sources and formats, defining a unified schema, implementing ingestion pipelines, applying necessary transformations, and utilizing Delta Lake's write operations for data persistence.