Databricks Certified Machine Learning - Associate
Get started today
Ultimate access to all questions.
Comments
Loading comments...
Ultimate access to all questions.
Given a large Spark DataFrame, you need to apply a function that involves complex pandas operations. How would you integrate these pandas operations into a Spark environment to ensure efficient processing?
A
Convert the entire Spark DataFrame to a pandas DataFrame and then apply the operations.
B
Use a Scalar Pandas UDF to apply the pandas operations row-wise in Spark.
C
Use a Grouped Map Pandas UDF to apply the pandas operations group-wise in Spark.
D
Use an Iterator Pandas UDF to apply the pandas operations in chunks.