You have a Databricks job that processes customer feedback data. The job has recently started failing due to an increase in feedback volume. Describe the steps you would take to scale the job to handle the increased load and ensure successful execution.

Simulated

Increase the number of worker nodes in the cluster, optimize the data processing logic for performance, and rerun the job.

83.3%

Reduce the scope of the data processing, keep the cluster size unchanged, and rerun the job.

2.3%

Manually split the feedback data into smaller batches, process each batch separately, and aggregate the results.

5.3%

Switch to a different cluster type with higher processing power, keep the data processing logic unchanged, and rerun the job.

9.0%

Databricks Certified Data Engineer - Professional