Answer-first summary for fast verification
Answer: Incorporate the shell script as an init script for faster execution
### Correct Answer: C - **Option A (Execute the shell script as the Workspace admin):** Incorrect. The script's execution time is not influenced by the user's administrative status. - **Option B (Utilize `%md` to accelerate the script execution):** Incorrect. `%md` is a magic command for Markdown, unrelated to shell script execution speed. - **Option C (Incorporate the shell script as an init script for faster execution):** Correct. Using `%sh` in a Databricks notebook confines execution to the driver node. For broader execution across worker nodes, embedding the script as an init script is essential. - **Option D (Augment the number of worker nodes to enhance script speed):** Incorrect. Since `%sh` commands only utilize the driver node, additional worker nodes won't affect execution speed. - **Option E (Trigger the notebook execution via the Databricks API):** Incorrect. The script's speed is independent of the notebook's triggering mechanism. **Additional Information:** Init scripts in Databricks can significantly optimize script execution across clusters.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
A data analyst is executing a shell script across all notebooks linked to a cluster, but the script's lengthy command set is causing significant delays. As a data engineer, what advice would you offer to expedite the process?
A
Execute the shell script as the Workspace admin
B
Utilize %md to accelerate the script execution
C
Incorporate the shell script as an init script for faster execution
D
Augment the number of worker nodes to enhance script speed
E
Trigger the notebook execution via the Databricks API
No comments yet.