Databricks Certified Data Engineer - Professional
Get started today
Ultimate access to all questions.
Comments
Loading comments...
Ultimate access to all questions.
When running a large Spark job that occasionally fails due to OutOfMemory errors, which technique can most effectively manage memory usage?
A
Using persist(StorageLevel.DISK_ONLY) for intermediate DataFrames._
B
Decreasing spark.executor.memory to force garbage collection to run more frequently.
C
Increasing the number of partitions to distribute data more evenly across executors.
D
Allocating more memory to the driver than executors to manage task distribution.