Ultimate access to all questions.
In a scenario where you are processing log data from multiple web servers using Spark Structured Streaming, how would you design your Spark job to handle schema drift and process time series data efficiently? Additionally, describe how you would manage data across partitions and within one partition to ensure optimal performance.