Databricks Certified Data Engineer - Professional
Get started today
Ultimate access to all questions.
Ultimate access to all questions.
In the context of designing a scalable and maintainable data pipeline for ingesting data from multiple sources into Delta Lake on Microsoft Azure, consider the following scenario: Your organization is expanding its data sources to include streaming data from IoT devices, batch data from on-premises databases, and real-time social media feeds. The pipeline must adhere to strict compliance standards, minimize costs, and be scalable to handle future data growth. Which of the following approaches BEST meets these requirements? Choose one option.
Explanation:
The correct answer is C because Azure Databricks, combined with Apache Spark and Delta Live Tables, provides a robust framework that supports scalable, maintainable, and cost-effective data ingestion from diverse sources. It offers built-in features for compliance, scalability, and cost management, making it the best choice for meeting the organization's requirements. Option A lacks flexibility and scalability, Option B increases maintenance complexity and misses out on the benefits of existing frameworks, and Option D neglects critical long-term considerations.