
Ultimate access to all questions.
How can you optimize a Spark Structured Streaming application for low-latency processing of financial transactions to ensure minimal processing time per micro-batch without compromising stateful accuracy?
A
Utilize trigger(Trigger.ProcessingTime("1 second")) and fine-tune spark.streaming.blockInterval.
B
Enable spark.streaming.receiver.maxRate and set a high spark.sql.streaming.metricsEnabled value.
C
Implement stateful transformations using mapGroupsWithState with a low trigger interval.
D
Opt for flatMapGroupsWithState with explicit state timeout settings and adjust spark.sql.streaming.schemaInference.