Design a data pipeline in Azure Data Factory that schedules the execution of Spark jobs on an Azure Databricks cluster. The pipeline needs to handle dynamic data inputs and adjust the cluster size based on the volume of data. Describe how you would manage the scheduling of these Spark jobs and the considerations for scaling the Databricks cluster.