Ultimate access to all questions.
Upgrade Now 🚀
Sign in to unlock AI tutor
In a Spark Structured Streaming application that involves complex stateful processing, which strategy best enhances scalability and fault tolerance in state management?
A
Implement stateless transformations wherever possible to minimize overhead.
B
Depend solely on checkpointing, storing state on the local filesystem for fault tolerance.
C
Employ stateful operators such as updateStateByKey, with checkpointing and write-ahead logs activated for recovery.
D
Opt for mapGroupsWithState for tailored state management, periodically saving state to an external database.
