Answer-first summary for fast verification
Answer: They can use clusters that are from a cluster pool
Using clusters from a cluster pool can significantly improve the start-up time for clusters. Cluster pools allow for faster and more efficient cluster creation because they maintain a set of idle, ready-to-use instances. This avoids the overhead associated with starting new instances from scratch, thereby reducing the overall cluster start-up time for jobs.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
A data engineer is responsible for managing a job that consists of multiple tasks and is scheduled to run every night. However, the execution of these tasks is slow due to the prolonged start-up time of the clusters. What measure can the data engineer take to decrease the start-up time for the clusters utilized in this nightly job?
A
They can use endpoints available in Databricks SQL
B
They can use jobs clusters instead of all-purpose clusters
C
They can configure the clusters to autoscale for larger data sizes
D
They can use clusters that are from a cluster pool
No comments yet.