Consider a scenario where you need to update the Python libraries on a Databricks cluster that is currently running a production machine learning model. The cluster uses the Databricks Runtime for Machine Learning and currently has TensorFlow 1.15 installed. A new version of TensorFlow, version 2.4, has been released and is required for a new project. Describe the steps you would take to update the TensorFlow library on this cluster while minimizing downtime and ensuring that the production model continues to run without interruption.

Simulated

Shut down the cluster, update TensorFlow to version 2.4, and restart the cluster.

7.3%

Create a new cluster with the updated TensorFlow 2.4, migrate the production model to the new cluster, and then decommission the old cluster.

65.9%

Update TensorFlow to version 2.4 on the existing cluster without stopping it, risking potential conflicts with the running production model.

7.3%

Install TensorFlow 2.4 in a separate environment and run the new project there, leaving the production model on the old TensorFlow version.

19.5%

Databricks Certified Machine Learning - Associate