AWS Certified Data Engineer - Associate
Get started today
Ultimate access to all questions.
Comments
Loading comments...
Ultimate access to all questions.
You are working on a data processing project that involves analyzing large volumes of clickstream data from a web application. The data includes user interactions, session information, and event metadata. Describe how you would use Apache Spark to create an ETL pipeline for this use case, and explain the considerations involved in handling high-velocity, high-volume data.
A
Use Apache Spark's batch processing capabilities to process the clickstream data at regular intervals, as real-time processing is not required.
B
Use Apache Spark Streaming to create a real-time ETL pipeline, with appropriate data sources, transformations, and sinks to handle the high-velocity, high-volume clickstream data efficiently.
C
Use a traditional database system to store and process the clickstream data, as it can handle high-velocity, high-volume data more effectively than Apache Spark.
D
Only process a subset of the clickstream data to reduce the volume and velocity, as real-time processing of the entire dataset is not feasible.