Answer-first summary for fast verification
Answer: (spark.table("sales").withColumn("avgPrice", col("sales") / col("units")).writeStream.option("checkpointLocation", checkpointPath).outputMode("append") .table("cleanedSales"))
The correct answer involves enriching the data by adding an average price column without performing any aggregations, which is characteristic of moving data from bronze to silver in the medallion architecture. Aggregations are typically performed when moving data from silver to gold. Understanding the role of each layer (bronze, silver, gold) in the medallion architecture is crucial for the Databricks Certified Data Engineer - Associate exam.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
Identify the Structured Streaming query that performs a hop from a bronze table to a Silver table by enriching the data without aggregation.
A
(spark.read.load(rawSalesLocation) .writeStream .option("checkpointLocation", checkpointPath) .outputMode("append") .table("uncleanedSales"))
B
(spark.table("sales").groupBy("store").agg(sum("sales")).writeStream.option("checkpointLocation",checkpointPath).outputMode("complete").table("aggregatedSales"))
C
(spark.table("sales").withColumn("avgPrice", col("sales") / col("units")).writeStream.option("checkpointLocation", checkpointPath).outputMode("append") .table("cleanedSales"))
D
(spark.readStream.load(rawSalesLocation).writeStream .option("checkpointLocation", checkpointPath) .outputMode("append") .table("uncleanedSales"))
E
(spark.table("sales").agg(sum("sales"),sum("units")).writeStream.option("checkpointLocation",checkpointPath).outputMode("complete").table("aggregatedSales"))