In the context of Pandas UDFs, explain the concept of broadcasting and its benefits when working with small control data in Spark. Provide an example of how you would use broadcasting in a Pandas UDF.

Simulated

Broadcasting is a technique where a small dataset is replicated across all nodes in a cluster to enable efficient distributed processing.

51.2%

Broadcasting is a technique where a small dataset is partitioned across multiple nodes in a cluster to enable parallel processing.

18.6%

Broadcasting is a technique where a small dataset is loaded into memory on a single node and accessed by all other nodes in the cluster.

20.9%

Broadcasting is not applicable when working with Pandas UDFs in Spark.

9.3%

Databricks Certified Machine Learning - Associate