
Ultimate access to all questions.
In the context of optimizing the performance of a Databricks cluster that is running multiple concurrent jobs, consider the following scenario: Your organization is facing increased costs due to high resource usage during peak hours, and there's a need to ensure compliance with data processing standards without sacrificing performance. Which of the following strategies would be the MOST effective in addressing these concerns while also improving the cluster's performance? Choose the best option.
A
Increase the number of worker nodes in the cluster to handle the increased workload, ensuring that all jobs have sufficient resources to run concurrently without contention.
B
Optimize the code within the jobs to improve efficiency and reduce resource usage, thereby lowering costs and maintaining compliance with data processing standards.
C
Schedule all jobs to run during off-peak hours to minimize resource contention and reduce costs, but this may delay data processing and reporting.
D
Disable autoscaling on the cluster to maintain a fixed number of worker nodes, which could lead to underutilization during low demand periods and overutilization during peak times.