Databricks Certified Data Engineer - Associate

Explanation

In the Databricks Lakehouse architecture:

• Bronze tables: Raw data ingestion
• Silver tables: Cleaned, validated data
• Gold tables: Aggregated, business-ready data for reporting and analytics

A "hop" from Silver to Gold table typically involves aggregation and transformation of cleaned data into business-ready aggregates. Let's analyze each option:

Option A: Reads from rawSalesLocation (likely Bronze) and writes to newSales - this is a Bronze to Silver hop.

Option B: Uses spark.read.load() (batch read) with writeStream - this is incorrect syntax and doesn't represent a Silver to Gold hop.

Option C: Reads from sales table (Silver), performs a simple transformation (avgPrice calculation), and writes to newSales - this is a transformation but not necessarily aggregation for Gold tables.

Option D: Reads from sales table (Silver), filters data, and writes to newSales - this is filtering but not aggregation.

Option E: Reads from sales table (Silver), performs aggregation (groupBy and sum), and writes to newSales with outputMode("complete") - this is a classic Silver to Gold hop where aggregated business metrics are created from cleaned data.

The key indicators of a Silver to Gold hop are:

1. Reading from an existing table (Silver layer)
2. Performing aggregation operations
3. Creating business-ready metrics
4. Using outputMode("complete") for aggregated results (common for Gold tables)

Therefore, Option E correctly represents a Structured Streaming query performing a hop from a Silver table to a Gold table.