
Explanation:
%md to accelerate the script execution): Incorrect. %md is a magic command for Markdown, unrelated to shell script execution speed.%sh in a Databricks notebook confines execution to the driver node. For broader execution across worker nodes, embedding the script as an init script is essential.%sh commands only utilize the driver node, additional worker nodes won't affect execution speed.Additional Information: Init scripts in Databricks can significantly optimize script execution across clusters.
Ultimate access to all questions.
No comments yet.
A data analyst is executing a shell script across all notebooks linked to a cluster, but the script's lengthy command set is causing significant delays. As a data engineer, what advice would you offer to expedite the process?
A
Execute the shell script as the Workspace admin
B
Utilize %md to accelerate the script execution
C
Incorporate the shell script as an init script for faster execution
D
Augment the number of worker nodes to enhance script speed
E
Trigger the notebook execution via the Databricks API