Answer-first summary for fast verification
Answer: DataFilters, PushedFilters, and the PartitionFilters are part of the Physical plan.
Logical plans in Spark are generated in a sequence starting with the Parsed or Unresolved Logical plan, followed by the Analyzed or Resolved Logical plan, and finally the Optimized Logical Plan. The Physical Plan serves as a bridge between Logical Plans and RDDs, incorporating various filters like DataFilters, PushedFilters, and PartitionFilters to optimize execution. The `explain` method allows viewing Physical plans, while Logical plans can be seen in the Spark UI. Push-down predicates, visible as PushedFilters in the Physical Plan, help avoid unnecessary data reads, enhancing performance.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
Which of the following statements accurately describes the relationship between Physical and Logical plans in Spark?
A
Logical plans and Physical plans can be viewed only in the Spark UI.
B
Physical plans can be viewed by using the explain method, whereas Logical plans can only be viewed in Spark UI.
C
After the submission of the spark application, the Physical plans are laid out followed by the generation of Logical Plans.
D
DataFilters, PushedFilters, and the PartitionFilters are part of the Physical plan.
E
Physical plans and Logical plans are only visible if the History server is enabled for Spark UI.
No comments yet.