Answer-first summary for fast verification
Answer: Utilize Azure Databricks with Apache Spark and Delta Live Tables to orchestrate and optimize the ingestion process, leveraging built-in scalability, compliance features, and cost-management tools.
The correct answer is C because Azure Databricks, combined with Apache Spark and Delta Live Tables, provides a robust framework that supports scalable, maintainable, and cost-effective data ingestion from diverse sources. It offers built-in features for compliance, scalability, and cost management, making it the best choice for meeting the organization's requirements. Option A lacks flexibility and scalability, Option B increases maintenance complexity and misses out on the benefits of existing frameworks, and Option D neglects critical long-term considerations.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
In the context of designing a scalable and maintainable data pipeline for ingesting data from multiple sources into Delta Lake on Microsoft Azure, consider the following scenario: Your organization is expanding its data sources to include streaming data from IoT devices, batch data from on-premises databases, and real-time social media feeds. The pipeline must adhere to strict compliance standards, minimize costs, and be scalable to handle future data growth. Which of the following approaches BEST meets these requirements? Choose one option.
A
Develop a monolithic application that uses a single, fixed method for ingesting all types of data, regardless of source characteristics or volume, to simplify initial development.
B
Create separate, custom ingestion scripts for each data source without utilizing any Azure or open-source data processing frameworks, to ensure complete control over the data flow.
C
Utilize Azure Databricks with Apache Spark and Delta Live Tables to orchestrate and optimize the ingestion process, leveraging built-in scalability, compliance features, and cost-management tools.
D
Focus solely on the immediate data ingestion needs using the simplest tools available, postponing any considerations for scalability, compliance, or cost optimization until the pipeline becomes operational.