Ultimate access to all questions.
You are working on a data processing project that involves analyzing large volumes of log data from various sources. The data includes structured and unstructured data, with varying formats and schemas. Describe how you would use Apache Spark to create an ETL pipeline that can handle the diverse data types and formats, and explain the steps involved in the process.