The data engineering team maintains a bronze table named 'orders_raw' where new orders data is appended nightly. They aim to create a Silver table, 'orders_cleaned', for a refined view of the orders data. What is the most cost-effective batch processing solution to propagate new records from 'orders_raw' to 'orders_cleaned'?

Real Exam

Utilize Delta Lake's time travel feature to compare the latest version of 'orders_raw' with its previous version, then write the differences to 'orders_cleaned'.

10.2%

Implement Spark Structured Streaming's foreachBatch logic with a 24-hour processing time trigger to process new records from 'orders_raw'._

14.0%

Reprocess all records in 'orders_raw' using batch overwrite logic to update 'orders_cleaned'.

6.5%

Apply an insert-only merge into 'orders_cleaned' using data from 'orders_raw' based on a composite key.

23.1%

Use Spark Structured Streaming to process new records from 'orders_raw' in batch mode with the trigger availableNow option._

46.2%

Databricks Certified Data Engineer - Professional

Get started today

Comments