Explanation

Correct Answer: C

Automated job clusters (also known as ephemeral clusters) are the most cost-efficient choice for periodic batch processing in Databricks. Because job clusters are provisioned only when the job starts and are terminated immediately upon completion, you only pay for the ~10 minutes of compute time (plus a small overhead for cluster startup) every hour. This prevents paying for idle compute time.

Analysis of Other Options:

• Structured Streaming (Trigger 60 min): A streaming job typically requires a cluster to remain active 24/7. Even with a long trigger interval, you would incur costs for the cluster while it sits idle between micro-batches.
• Interactive Clusters: These are intended for ad-hoc analysis and are priced at a higher rate than job clusters. Using a dedicated interactive cluster for an hourly job results in 50 minutes of wasted idle time every hour at a premium price.
• Manual Triggers: This is not a scalable or reliable way to meet a production SLA, as it introduces human error and lacks automation.
• Event-driven (File Arrival): This does not guarantee an hourly cadence. If files arrive irregularly (e.g., every 5 minutes or every 2 hours), the dashboard will either update too frequently (increasing costs) or too slowly (violating the SLA).