Answer-first summary for fast verification
Answer: Change the cluster from all-purpose to job cluster when scheduling the job
Using an all-purpose cluster is acceptable during development, but for scheduled jobs where interaction with the notebook isn't necessary, switching to a job cluster can significantly reduce costs. The difference in cost arises from the DBU (Databricks Unit) charges, not the compute costs from the cloud provider. Job clusters are less expensive in terms of DBU costs compared to all-purpose clusters. For example, AWS charges $0.15 per DBU for job clusters versus $0.55 per DBU for all-purpose clusters. To check the DBU cost for your cluster, look at the cluster details in the top right corner.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
A team member is using an all-purpose cluster for both developing a notebook and setting up a job that runs every 30 minutes to update tables used in a dashboard. What recommendation would you make to reduce the overall cost of this approach?
A
Reduce the size of the cluster
B
Reduce the number of nodes and enable auto scale
C
Enable auto termination after 30 mins
D
Change the cluster from all-purpose to job cluster when scheduling the job
E
Change the cluster mode from all-purpose to single-mode
No comments yet.