Databricks Certified Data Engineer - Professional
Get started today
Ultimate access to all questions.
Comments
Loading comments...
Ultimate access to all questions.
When joining a large DataFrame df1 with a small DataFrame df2 in Spark, what is the most efficient method to optimize the operation?
A
Repartition df1 to match the number of partitions in df2 before joining.
B
Convert df2 to an RDD and manually broadcast it for the join operation with df1.
C
Cache both DataFrames in memory before performing the join to speed up access.
D
Use broadcast variables to distribute df2 and apply the broadcast hint in the join operation.