Answer-first summary for fast verification
Answer: Use the Databricks Library management system to install the library at the cluster level, ensuring it's available to all users.
Option C is the most efficient approach as it centralizes the library installation process at the cluster level, making it available to all users without the need for individual installations. This also ensures consistency across the environment.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
As a data engineer working with Databricks, you are tasked with optimizing the setup for a machine learning project. You need to create a cluster with the Databricks Runtime for Machine Learning and ensure that a specific Python library is available for all users. What is the most efficient way to achieve this?
A
Create a custom Databricks Runtime for Machine Learning with the library pre-installed and deploy it.
B
Manually install the library in each user's notebook using a pip command.
C
Use the Databricks Library management system to install the library at the cluster level, ensuring it's available to all users.
D
Instruct each user to install the library in their local development environment before pushing the code to Databricks.