Answer-first summary for fast verification
Answer: Utilize a Cloud Dataproc Workflow Template to manage and execute the Spark jobs efficiently.
The optimal solution is to use a Cloud Dataproc Workflow Template, which allows for the automation of Spark job sequences and concurrent executions. This approach is complemented by integrating with Apache Airflow DAGs via Cloud Composer for scheduled triggers. Reference: [Google Cloud Dataproc Workflow Templates](https://cloud.google.com/dataproc/docs/tutorials/workflow-composer).
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
You have multiple Spark jobs scheduled to run on a Cloud Dataproc cluster, with some jobs running in sequence and others concurrently. What is the best method to automate this process?
A
Develop a Bash script utilizing the Cloud SDK to create a cluster, execute jobs, and terminate the cluster afterwards.
B
Construct a Directed Acyclic Graph (DAG) in Cloud Composer for job orchestration.
C
Implement an initialization action to automatically run the jobs upon cluster creation.
D
Utilize a Cloud Dataproc Workflow Template to manage and execute the Spark jobs efficiently.