Answer-first summary for fast verification
Answer: Lazy evaluation in native Spark DataFrames means that transformations are executed only when an action is called, while in Pandas API on Spark, transformations are executed immediately.
Lazy evaluation in native Spark DataFrames means that transformations are not executed immediately but are instead optimized and executed only when an action is called, such as `collect()` or `show()`. This allows Spark to optimize the query execution plan. In contrast, Pandas API on Spark does not support lazy evaluation, and transformations are executed immediately, which can lead to less efficient query execution for large datasets.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
Explain how lazy evaluation works in native Spark DataFrames and how it differs from the evaluation strategy in Pandas API on Spark. Provide a detailed example to illustrate the differences.
A
Lazy evaluation in native Spark DataFrames means that transformations are executed immediately, while in Pandas API on Spark, transformations are executed only when an action is called.
B
Lazy evaluation in native Spark DataFrames means that transformations are executed only when an action is called, while in Pandas API on Spark, transformations are executed immediately.
C
Lazy evaluation in both native Spark DataFrames and Pandas API on Spark means that transformations are executed immediately.
D
Lazy evaluation in both native Spark DataFrames and Pandas API on Spark means that transformations are executed only when an action is called.