Answer-first summary for fast verification
Answer: Configure the session timeout to be 25 minutes.
The 15-minute delay is caused by Spark session initialization overhead. Since sessions run every 40 minutes and each session lasts 15 minutes, there's a 25-minute gap between sessions. By configuring the session timeout to 25 minutes (option A), the Spark session remains active between runs, eliminating the initialization delay. This approach keeps the session 'warm' and ready for the next execution. Option B (increasing nodes) doesn't address session initialization delays. Option C (dynamic executors) helps with resource optimization but not session startup. Option D (isolated compute) provides security isolation but doesn't reduce session initialization time. The community discussion shows 100% consensus on option A with the reasoning that keeping the session active between runs prevents shutdown and reinitialization delays.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
You have an Azure Machine Learning workspace and a serverless Spark compute. You plan to run a Spark job in a session that repeats every 40 minutes, with each session lasting 15 minutes. During testing, you notice a 15-minute startup delay for the Spark sessions. You need to reduce this startup delay to under one minute. What should you do?
A
Configure the session timeout to be 25 minutes.
B
Increase the number of nodes to 16.
C
Enable dynamically allocated executors.
D
Enable an isolated compute.