
Ultimate access to all questions.
In a scenario where you are optimizing a Databricks environment for a machine learning pipeline, you need to ensure that the Databricks Runtime for Machine Learning is configured to handle the specific computational requirements of your models. Additionally, you must install a Python library that supports distributed processing. Which of the following steps should you take to achieve this?
A
Create a cluster with the Databricks Runtime for Machine Learning, and manually install the distributed processing library in each notebook.
B
Use the Databricks Runtime for Machine Learning with autoscaling enabled, and install the distributed processing library using the Library management system at the cluster scope.
C
Create a custom Databricks Runtime for Machine Learning that includes the distributed processing library in its base image.
D
Instruct team members to install the distributed processing library in their local development environments and synchronize the installations across all notebooks.