Explanation:
Setting a state retention policy and using checkpointing helps in managing the state store size and ensuring fault tolerance. The mapGroupsWithState function is used to manage stateful operations efficiently.
Ultimate access to all questions.
No comments yet.
In a scenario where you need to explore and tune state information using stream-static joins and Delta Lake, describe the steps you would take to optimize the state store. Include considerations for state retention, checkpointing, and the use of mapGroupsWithState.
A
Use mapGroupsWithState without setting a state retention policy and avoid checkpointing.
B
Set a state retention policy, use checkpointing to ensure fault tolerance, and optimize the state store by using mapGroupsWithState.
C
Ignore state retention and checkpointing, and directly perform stream-static joins.
D
Use a batch query to avoid dealing with state information.