Databricks Certified Data Engineer - Associate
Comments
Loading comments...
Ultimate access to all questions.
In a Databricks environment, you are tasked with managing a project that utilizes multiple libraries and dependencies across various notebooks and clusters. The project requires high consistency and reproducibility to ensure that all team members work in the same environment, and to avoid any compatibility issues. Considering the need for scalability, ease of management, and minimizing manual errors, which of the following approaches is the BEST for managing these dependencies? Choose one option.
A
Manually installing each required library and dependency on every cluster used in the project, ensuring that each cluster's environment is configured identically.
B
Utilizing Databricks' built-in library management features to attach the necessary libraries to each notebook individually, allowing for notebook-specific dependency management.
C
Developing a custom Docker image that includes all the required libraries and dependencies, then using this image as the base for all clusters within the project to ensure a uniform environment.
D
Embedding the installation commands for all necessary libraries and dependencies within each notebook, executing these commands at the start of each notebook run to setup the environment.