Databricks Certified Data Engineer - Professional
Get started today
Ultimate access to all questions.
Comments
Loading comments...
Ultimate access to all questions.
In a Databricks environment, you are working on a project that involves multiple Python sub-modules currently maintained as Wheels. The project requires adapting these Wheels to use direct imports with relative paths in Databricks notebooks to improve maintainability and scalability. Considering the need for a proper package structure, cost efficiency, and compliance with Python packaging standards, which of the following approaches would you choose? (Choose one option)
A
Modify the import statements in each notebook to use absolute paths based on the location of the Wheels in the Databricks file system, ensuring quick access but potentially complicating future updates.
B
Refactor the Python code to include a top-level package with an __init__.py file, and update the import statements in the notebooks to use relative paths based on the new package structure, adhering to Python packaging standards.
C
Use the Databricks workspace's shared storage to host the Wheels, and update the import statements in the notebooks to reference the Wheels directly, which may increase storage costs but simplifies initial setup.
D
Create a new library in Databricks for each Wheel and use the library's URI in the notebook to import the required modules, which could lead to management overhead but ensures isolation of dependencies.