Answer-first summary for fast verification
Answer: In-Memory Computation
In Spark, leveraging in-memory computation is a key feature for optimizing performance. This technique involves caching or persisting intermediate data in memory, which is especially beneficial for iterative machine learning algorithms. By storing data in memory, Spark minimizes the need for disk I/O, leading to faster execution times for tasks that require multiple iterations over the same dataset.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
In the context of optimizing Spark performance for a large-scale machine learning project, which technique is used to store intermediate data in memory, thereby speeding up iterative algorithms and reducing disk I/O?
A
Data Shuffling
B
Disk Caching
C
In-Memory Computation
D
Data Replication
No comments yet.