Databricks Certified Data Engineer - Professional
Comments
Loading comments...
Ultimate access to all questions.
A team of data engineers is collaborating on notebooks attached to a shared cluster. The admin initially set the Spark config property spark.sql.autoBroadcastJoinThreshold to 100b during cluster creation. Later, a data engineer in the show_orders notebook changes this property to 1024b using spark.conf.set. Subsequently, another data engineer in the fetch_results notebook runs %run ./show_orders followed by retrieving the property's value. What will be the output of this retrieval?_
A
The output will be 1024b because the notebook-level configuration overrides the cluster-level setting.
B
The output will be 100b as the cluster-level configuration cannot be modified by notebook-level settings.
C
The output will be 100b since changes in one notebook do not affect another notebook's configuration.
D
There will be no output due to the absence of a print statement.
E
The output will be None because the property's default value is None.