Microsoft Azure Data Engineer Associate - DP-203
Get started today
Ultimate access to all questions.
Ultimate access to all questions.
You are tasked with designing an incremental data load strategy for a large e-commerce company that experiences daily fluctuations in sales data. The company wants to ensure that the data warehouse is updated with only the new transactions each day. Describe the steps you would take to implement this strategy using Azure Data Factory, including the use of watermarks and the integration of Azure SQL Database as the source system.
Explanation:
The correct approach is to use a watermarking strategy, which involves identifying the maximum transaction ID or timestamp from the previous load and using this as a filter for the next load. This ensures that only new transactions are loaded into the data warehouse, optimizing performance and resource usage.