Answer-first summary for fast verification
Answer: Set up Amazon EventBridge to monitor RDS for new transactions, trigger AWS Glue jobs for transformation, and use AWS Step Functions to orchestrate the workflow, ensuring data is loaded into Redshift efficiently.
Option B is the most efficient and scalable solution. It leverages event-driven architecture with EventBridge to automate the ETL process, uses Glue for data transformation, and Step Functions for orchestration, ensuring a serverless and scalable workflow.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
You are tasked with designing an ETL pipeline for a large e-commerce company that processes millions of transactions daily. The pipeline needs to extract data from Amazon RDS, transform it using AWS Glue, and load it into Amazon Redshift for analytics. Describe how you would integrate these services to create a robust ETL pipeline, considering event-driven architecture and serverless workflows.
A
Use AWS Lambda to trigger the ETL process directly from RDS, bypassing Glue and loading data directly into Redshift.
B
Set up Amazon EventBridge to monitor RDS for new transactions, trigger AWS Glue jobs for transformation, and use AWS Step Functions to orchestrate the workflow, ensuring data is loaded into Redshift efficiently.
C
Manually run AWS Glue jobs periodically to extract data from RDS, transform it, and then use a script to load the transformed data into Redshift.
D
Use Amazon S3 as an intermediary storage, dump data from RDS to S3, trigger Glue jobs from S3 events, and then load data from S3 to Redshift.