Databricks Certified Data Engineer - Professional
Get started today
Ultimate access to all questions.
Comments
Loading comments...
Ultimate access to all questions.
Describe the use case and benefits of using repartitionByRange in PySpark. How does this method differ from repartition and coalesce in terms of data distribution and performance?
A
repartitionByRange is used for evenly distributing data across partitions without considering the order of data, similar to repartition.
B
repartitionByRange ensures that data is partitioned based on the range of values in a specified column, which can be beneficial for optimizing queries that filter based on ranges.
C
repartitionByRange is similar to coalesce in that it reduces the number of partitions but does not shuffle data, making it less efficient for range-based queries.
D
repartitionByRange should be avoided as it always leads to uneven data distribution, making it less efficient than repartition.