Explanation:
This question assesses your understanding of the Databricks REST API, specifically the 2.0/jobs/create endpoint. The correct JSON configuration must use spark_python_task for Python script execution, specify the cluster by its ID (not name), and correctly pass parameters as an array. The job name is optional. The correct configuration is:
{
"existing_cluster_id": "1198-132537-dht25rtr",
"spark_python_task": {
"python_file": "dbfs:/fetch_matches.py",
"parameters": ["2019", "11"]
}
}
{
"existing_cluster_id": "1198-132537-dht25rtr",
"spark_python_task": {
"python_file": "dbfs:/fetch_matches.py",
"parameters": ["2019", "11"]
}
}
Exam Tip: Remember, the job name is optional, and the cluster must be specified by its ID when using an existing cluster.
Ultimate access to all questions.
A data engineer is tasked with creating a new job in Databricks to execute a Python script located at dbfs:/fetch_matches.py. This script requires year and month as command-line arguments to fetch soccer matches data for the specified period. Given an existing cluster named matches with ID 1198-132537-dht25rtr, which of the following JSON configurations correctly creates the job for the year 2019 and month 11 via the 2.0/jobs/create endpoint?
A
{ "name": "Get All Matches", "existing_cluster": "matches", "python_task": { "python_file": "dbfs:/fetch_matches.py", "parameters": ["2019", "11"] } }
B
{ "name": "Get All Matches", "existing_cluster_id": "1198-132537-dht25rtr", "spark_python_task": { "python_file": "dbfs:/fetch_matches.py", "arguments": "["2019", "11"]" } }
C
{ "existing_cluster_id": "1198-132537-dht25rtr", "spark_python_task": { "python_file": "dbfs:/fetch_matches.py", "parameters": ["2019", "11"] } }
D
{ "existing_cluster": "matches", "spark_python_task": { "python_file_path": "dbfs:/fetch_matches.py", "arguments": ["2019", "11"] } }
E
{ "name": "Get All Matches", "existing_cluster_id": "1198-132537-dht25rtr", "python_task": { "python_file_path": "dbfs:/fetch_matches.py", "parameters": "["2019", "11"]" } }
No comments yet.