
Ultimate access to all questions.
You are tasked with creating an ETL pipeline for a financial services company that needs to integrate data from multiple sources, including transaction data, customer feedback, and market data. The company requires real-time analytics to make quick business decisions. Describe how you would design this ETL pipeline, including the tools and technologies you would use, and explain how you would handle the challenges posed by the volume, velocity, and variety of data.
A
Use traditional SQL-based ETL tools and batch processing for all data sources.
B
Leverage AWS Glue for ETL jobs, Amazon Kinesis for real-time data streaming, and Amazon Redshift for data warehousing.
C
Only use Apache Kafka for data ingestion and ignore structured data processing.
D
Manually script ETL processes using Python without using any cloud services.