Answer-first summary for fast verification
Answer: Use Amazon S3 event notifications to trigger AWS Glue jobs directly for data transformation and loading into Redshift.
Option A is the most efficient solution for handling large volumes of data. It leverages S3 event notifications to directly trigger Glue jobs, which process and load data into Redshift, ensuring a scalable and efficient pipeline.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
You are designing a data pipeline for a retail company that needs to analyze sales data. The pipeline must extract data from Amazon S3, transform it using AWS Glue, and load it into Amazon Redshift. How would you configure this pipeline to handle large volumes of data and ensure efficient processing?
A
Use Amazon S3 event notifications to trigger AWS Glue jobs directly for data transformation and loading into Redshift.
B
Set up a cron job on an EC2 instance to periodically check S3 for new files and then trigger Glue jobs.
C
Manually run AWS Glue jobs to extract data from S3, transform it, and then load it into Redshift.
D
Use Amazon SQS to queue S3 events and have Glue jobs poll the queue for processing.
No comments yet.