Answer-first summary for fast verification
Answer: from databricks import jobs # Data preprocessing job data_preprocessing_job = jobs.create("data_preprocessing", "/path/to/data_preprocessing/notebook") # Feature engineering job feature_engineering_job = jobs.create("feature_engineering", "/path/to/feature_engineering/notebook") feature_engineering_job.run(after=data_preprocessing_job) # Model training job model_training_job = jobs.create("model_training", "/path/to/model_training/notebook") model_training_job.run(after=feature_engineering_job) # Model evaluation job model_evaluation_job = jobs.create("model_evaluation", "/path/to/model_evaluation/notebook") model_evaluation_job.run(after=model_training_job)
The correct code snippet demonstrates how to create a Databricks job for each task and chain them together to form a multi-task ML workflow. It uses the 'after' parameter to specify the dependency between jobs, ensuring that each job runs after the completion of the previous job. Option D correctly implements this chaining mechanism, while options A and B do not specify any dependencies between jobs, and option C has an incorrect syntax for the 'after' parameter.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
You are working on a machine learning project that involves multiple tasks, such as data preprocessing, feature engineering, model training, and model evaluation. You need to orchestrate these tasks using Databricks jobs. Write a code snippet that demonstrates how to create a Databricks job for each task and chain them together to form a multi-task ML workflow.
A
jobs = dbutils.notebook.entry_point()
data_preprocessing_job = jobs.create("data_preprocessing", "/path/to/data_preprocessing/notebook") data_preprocessing_job.run()
feature_engineering_job = jobs.create("feature_engineering", "/path/to/feature_engineering/notebook") feature_engineering_job.run()
model_training_job = jobs.create("model_training", "/path/to/model_training/notebook") model_training_job.run()
model_evaluation_job = jobs.create("model_evaluation", "/path/to/model_evaluation/notebook") model_evaluation_job.run()
B
from databricks import jobs
data_preprocessing_job = jobs.create("data_preprocessing", "/path/to/data_preprocessing/notebook") data_preprocessing_job.run()
feature_engineering_job = jobs.create("feature_engineering", "/path/to/feature_engineering/notebook") feature_engineering_job.run()
model_training_job = jobs.create("model_training", "/path/to/model_training/notebook") model_training_job.run()
model_evaluation_job = jobs.create("model_evaluation", "/path/to/model_evaluation/notebook") model_evaluation_job.run()
C
from databricks import jobs
data_preprocessing_job = jobs.create("data_preprocessing", "/path/to/data_preprocessing/notebook") data_preprocessing_job.run()
feature_engineering_job = jobs.create("feature_engineering", "/path/to/feature_engineering/notebook") feature_engineering_job.run()
model_training_job = jobs.create("model_training", "/path/to/model_training/notebook") model_training_job.run(after=[feature_engineering_job])
model_evaluation_job = jobs.create("model_evaluation", "/path/to/model_evaluation/notebook") model_evaluation_job.run(after=[model_training_job])
D
from databricks import jobs
data_preprocessing_job = jobs.create("data_preprocessing", "/path/to/data_preprocessing/notebook")
feature_engineering_job = jobs.create("feature_engineering", "/path/to/feature_engineering/notebook") feature_engineering_job.run(after=data_preprocessing_job)
model_training_job = jobs.create("model_training", "/path/to/model_training/notebook") model_training_job.run(after=feature_engineering_job)
model_evaluation_job = jobs.create("model_evaluation", "/path/to/model_evaluation/notebook") model_evaluation_job.run(after=model_training_job)