Option C is the most effective method for long-term performance monitoring of Azure Databricks pipelines. By integrating Azure Monitor with Application Insights, you can continuously collect and analyze performance metrics from Databricks. This setup facilitates the tracking of essential performance indicators such as run times and memory usage over time. Dashboards in Azure Monitor offer a straightforward way to visualize trends and spot potential issues. Moreover, configuring alerts for notable changes in performance metrics enables proactive issue resolution. This approach ensures a thorough and streamlined solution for overseeing data pipeline performance over extended periods.