Answer: Pin the cluster.

## Analysis of the Question The requirement is to retain an Azure Databricks interactive cluster's configuration indefinitely after termination while minimizing costs. The cluster is used infrequently and has auto-termination enabled. ## Evaluation of Options ### **A: Pin the cluster** ✅ - **Optimal Choice**: Pinning a cluster in Azure Databricks preserves the cluster configuration permanently, even after termination. - **Cost Efficiency**: Pinning doesn't keep the cluster running; it only retains the configuration metadata. The cluster still terminates when inactive, avoiding compute costs. - **Best Practice**: This is the standard Azure Databricks feature designed specifically for preserving cluster configurations long-term without ongoing expenses. ### **B: Create an Azure runbook that starts the cluster every 90 days** ❌ - **Inefficient**: This approach would incur unnecessary compute costs by periodically starting the cluster. - **Complexity**: Requires additional Azure Automation resources and maintenance. - **Not Cost-Effective**: Regular cluster startups defeat the purpose of minimizing costs for infrequent usage. ### **C: Terminate the cluster manually when processing completes** ❌ - **Configuration Loss**: By default, Azure Databricks only retains terminated cluster configurations for 30 days for all-purpose clusters. - **Doesn't Meet Requirement**: This approach doesn't ensure indefinite retention of the configuration. ### **D: Clone the cluster after it is terminated** ❌ - **Temporary Solution**: Cloning creates a new cluster but doesn't preserve the original configuration indefinitely. - **Manual Process**: Requires intervention each time the configuration needs to be reused. - **Not Permanent**: The cloned cluster would also be subject to the same retention limitations. ## Why Pinning is the Optimal Solution Pinning is specifically designed for this scenario: - **Configuration Preservation**: Pinned clusters retain their configuration metadata permanently in Azure Databricks. - **Cost Optimization**: No compute costs are incurred while the cluster is terminated; only configuration storage. - **Ease of Use**: When needed again, the pinned cluster can be easily restarted with the exact same configuration. - **Azure Databricks Best Practice**: This is the recommended approach for preserving cluster configurations for infrequently used clusters.

Author: LeetQuiz Editorial Team