Ultimate access to all questions.
Upgrade Now 🚀
Sign in to unlock AI tutor
A data engineering team is working on ETL pipelines using a shared Databricks cluster and needs to utilize a third-party Python library, etl_utils, in their notebooks. What is the best method to ensure this library is accessible to all team members?
etl_utils
A
Modify the cluster to utilize the Databricks Runtime for Data Engineering.
B
Execute %pip install etl_utils in any notebook connected to the cluster.
%pip install etl_utils
C
Adjust the PYTHONPATH variable in the cluster settings to include the path to etl_utils.
PYTHONPATH
D
Incorporate the dbutils.library.installPyPI('etl_utils') command into the cluster's initialization script.
dbutils.library.installPyPI('etl_utils')
E
It's impossible to make the etl_utils library available on a cluster.
