Databricks Certified Data Engineer - Professional

Explanation

In Databricks Delta Live Tables (DLT) pipelines, the correct way to get configuration variables is using spark.conf.get().

Here's why:

• spark.conf.get() - This is the standard Apache Spark method for accessing configuration parameters. In DLT pipelines, configuration variables are stored in Spark configuration and can be accessed using this method.

• spark.get() - This is not a valid method in Spark for accessing configuration variables.

• @dlt.get() - This is not a valid decorator or method in DLT pipelines.

• @dlt.spark.conf.get() - This syntax is incorrect as @dlt decorators are used for defining tables and views, not for accessing configuration.

In DLT pipelines, you typically use configuration variables like this:

my_config_value = spark.conf.get("my.config.key")

my_config_value = spark.conf.get("my.config.key")

This allows you to parameterize your DLT pipelines and make them more flexible for different environments.